 Okay, welcome everyone I'm going to talk to you today about telemetry projects what we think we're going to do in the next cycle so I'm Julian Gordon is very and is going to help me to run through this slide as I need so quick summary of telemetry what it does I hope everyone knows by now because we are getting pretty old as a open stack project so we do metric stuff in open stack we gather metrics we store them and we try to provide them with the useful data to do any kind of magic like I don't know billing capacity planning prediction any kind and alarming we are made of free service so that used to be for that the news is that one of our service left open stack that would be gnocchi so the main services that remains in telemetry are Cilometer the last one which is responsible as of now to gather the metrics from all the open stack components and put them into some data store which is by G4 gnocchi hey is the alarming engine which is responsible to either do things based on the whatever threshold has been reached by a metric or execute an action based on an event and notifications and by any of the other open stack components and the last piece is panko which is the service that store the events the flex of events coming from the different open stack services into a database and providing an API to access them so the project was funded during the cruise release so I'm lazy I use the template provided by the foundation which put a lot of nice information on that so pretty cool we're officially I counted we're 50 contributors for the last cycle that's the number of people that committed something that I counted but we are actually free during 80% of the work so since a few cycles we have less and less people contributing to open stack mainly due to the amount of people sent by a different company that left open stack recently during the last cycle I'm sure everyone is aware of that so it's mainly managed by me Gordon and media back who was not there today the last resource of adoption number I don't have any comment on that it's a number reported by the foundation of our user survey it's in that a lot of people are using it but we don't have much information on all and why so if you want to give us some insight it's a good time to do this at the end of the talk so the new feature and it's one for five so it's pretty hard to for us to say anything about that because we are free and we don't always plan a way ahead what we're going to do because when you have only a few developers to maintain four projects print and mh3 plus new key where we do spend a lot of time it's pretty hard to gather in advance requirements and things you're going to do so one thing we've done so far for this cycle is to deprecate the collector of thermometer so the good thing is that it's going to reduce a lot the load that you see on your rabbit mq if you use that as a back end it's currently all the messages or many of the messages passes on rabbit mq maybe twice if if the information comes from notification so we're going to be stopped by this if you stop using the collector so now the agents are going to put directly the data into whatever back end you use and not go for the collector so it will be removed in the next cycle if nobody complains about that we added a new zhaka publisher so and the same spirit not going through the collector we use directly publisher to publish data from the agents pulling the data from open stack to something else here it's a car so that allows you to do things like grab the flecks of events from any open stack components nova notron cinder whatever that goes through the rabbit mq and that's grabbed by cinameter and push into a queue into zhaka so then your application or your operator application more likely can consume the zhaka feed of events as I said we used to offer projects in telemetry before being gnocchi which left a couple of people go it's still a bit there we're in the process of moving into its own github organization we moved it for one of the biggest reasons that gnocchi works pretty well with open stack it uses swift for storage if you want you can use keystone for notification but it's pretty agnostic and pretty generic as it is a term series database so it doesn't have anything really open stack specific in except that it was born in the open stack ecosystem and grew there but in order to facilitate the adoption outside of open stack of the project and to maybe grow its number of contributors we decided to move it out of open stack and see what will happen so I had to pick themes for the telemetry actually so I randomly pick a few things I think we're trying to focus but I don't know what you think Gordon does that look looks good is this is what you're going to do during this cycle yeah not focus you're good at that okay so we're going to focus on a lot of things we are I mean I always put like major focus on on UX because this is something we try to do like we're in one of those project range you have good defaults like by default if you want to install gnocchi or cilometer it has to work I mean we like our depth stack plugin is pretty small because we don't configure a lot of things in depth stack you just have to install it and even without any configuration file it ought to work by default I'm not always always through but it should be we don't we don't put a lot of focus on scalability I think in this release we improve a lot of things with twos the way we do the balance between the jobs in the different collectors agents in and cilometer but that's mainly things you can see as a end user or maybe as a operator because it would be more solid when you deploy it will have less likely to be to crash or anything but it's there's not a lot of user or operator visible feature we do in visual is in cilometer mainly because the project is a bit complete in term of features so far and also because we are we have just enough resources to maintain the project and to reduce the technical depth so putting new features and new things to manage is a bit out of our scope in a lot of project like AI or cilometer same thing for queens I think we're going to move a bit from the resiliency to the scalability mainly we did we did a through that I did a talk on Tuesday about cilometer and yokey and run using them to pull and make meters 10,000 instances so we did that with Alex Cruz from Red Hat to see how far can go cilometer and yokey and we did it a few bottlenecks which is why I think it would be too late for pike and the before queens will be off time and resources to take all a few of the scalability issue we might have in cilometer itself to pull like it can be pretty slow to pause and when you have a lot of instances or any resources like networks or volumes of whatever to pause it might be a bit slow for cilometer to do and to generate so right amount of events in the good time frame these are the like I said the possible feature we might work so the better scalability like I said for the agent we might be a bit a bit slow there's a lot of work to do for anyone interested in A or alarming engine it's pretty simple it works it does things like compare threshold against metrics and raise alarms or whatever but it's not that efficient in the end there's a lot of of optimizations that could be done I think we know that for a long time but we don't have any resources or developer working on that for the last cycles so that's not very good I'm not sure we'll have time and resources to work on that but that's something we have in mind like I said UX pretty important for us so we have a long list of things to do to improve the documentation like when we started gnocchi three years ago we had this no doc no merge policy which finally worked pretty well because the project is the best documented one we have in telemetry the doc is pretty accurate everything is explained as a doc every time somebody had a feature in gnocchi it has to write the documentation with it so it allows us to make sure that the feature is easy to be consumed by the user and that the documentation is properly written by someone who wrote the code so we can compare both and say the doc is wrong or the code is wrong but we don't do that yet for the over telemetry projects such as cinematometer because there's such a huge gap between the state of the project and the documentation which probably covers only a small part of cinematometer but it's pretty hard to say to developers your feature is nice but you need to write the doc when the doc is pretty empty so we don't we need to catch up on that it's a long process to do or someone needs to just start watching the non-existing documentation and rework everything so we move the install guide I think last cycle into the repositories that's the first step we need to merge them in one single documentation at some point yeah yeah so we have the same information twice the install guide the doc team because the way it was back then it was the doc team was written responsible to write the documentation and the developer was supposed to write the code that doesn't work very well so now we need to merge back everything into world documents which is a huge task that nobody is taking care of really I think so far except you like from time to time you you move things around but it's yeah error release themes no idea but from the bottom that's better I mean it's pretty hard to to know what we do in this cycle in next cycle so in two cycles it's pretty hard to to know anything so this is just a border plate don't pay that I left here but I really don't know what we're going to do so we need you up I think that's pretty obvious if you want telemetry to to continue to to free them to have new features and fancy stuff we definitely need more resources I mean the project is not going to disappear anytime soon but but it's pretty hard to have big things to announce every summit every six months when you only have an handful of people walking on the project we also like to think back so we know that a lot of people are actually using things like Cinometer and your key but it's pretty hard to gather feedback we we expect and we hoped to to that week has a summit would have been a good opportunity to gather feedback so far I did have some feedback from a few people operating new key and Cinometer at scale which was pretty interesting but we always glad to hear about that here during the summit or thought anytime I mean we use the open stack dev ending list and we were pretty easy to reach so if you have anything you want to ask us or to discuss or to learn about telemetry we had an onboarding session earlier this week which was not really crowded but I mean we do onboarding all the year so don't worry if you want to just show up at some point and help us we'll be happy to to have you onboard and this is it it's a small project so not a long presentation anything you want to add Gordon before our site I'll start asking who wants question sure you want to take the mic so the that polling stuff you're mentioning when you did the benchmarks with Alex yeah was that with the advert polling stuff that we added or was that against no API oh for the polling many added the feature where you can instead of polling no API I don't think it was used in this case okay yeah because it was better hopefully better we can try test that against benchmark what like the new bench like if we did a subsequent benchmark we would probably use the advert polling instead of yeah I mean we need to use it but it was not available when we did the test with Alex so it was upstream and master but the version we used so I don't think I don't think it was a set I checked with Alex but we might need to do more benchmark on thermometer without option any more question yep sure I primarily use it as a source of information for another monitoring system so we'll just let it do the collection and that's sort of how we view it originally implemented that around ice house time frame or maybe yeah around there and at the time the only way to get the data out of it both events and meters was to write a dispatcher plugin and actually we still have that today although at some point it split into two plugins but I guess dispatchers it sounds like we'll probably not be a thing moving forward yeah so the dispatcher are part of the collector but we just deprecated so the new way of pushing things is not to go through the dispatcher which if you go through the dispatcher you have to go through the agent to the dispatcher of the rabbit mq which is usually a problem so we just decided to deprecate that so if you have a dispatcher code it's pretty easy to adapt it to be no a publisher so publisher is exactly like a dispatcher except that it's run by the agent directly so it's just a layer above of what used to be the collector so it's not a big deal you just have to adapt your plugin to that that's fine we're kind of planning on that um when when we have to do that when is so uh and this bike release it's going to be more cost deprecated so your dispatcher will still work and unless we hit something big I think we can remove it in one cycle two cycle I never remember how long is the deprecating period like it's two cycles okay so we might remove it in the next cycle we don't really care if we remove it or not it's not like a big piece of code we have to maintain and there's no like bugs or anything in it so it's not there are some bugs like memory consumption which can be pretty heavy or better people that deploy it since it's deprecated we can just say it's a we know it's a problem just it's deprecated so stop using it so we might wait two cycles to remove it we're not really in a hurry or anything but it's not all the same functionality now in the publisher I imagine yeah it's exactly the same thing because that was the only issue in the past like was the events were missing from that so um I kind of had to do it that way yeah but no it's it's it's also a publisher for events so a lot of times I wasn't 100 clear what process these things were going to be evaluated and you know so it's a little bit of work there but it sounds like the architecture is a lot simpler really forward yep good anyways thank you