 So welcome to the PTL webinar series. These series evolved from sessions that were held by the PTLs regarding updates to their projects at each summit. We've converted those talks into webinars to extend the reach of the events beyond this summit. And today our PTLs that are going to present will be John Griffiths with Block Storage and Owen Glenn with Telemetry. So if you have questions, you can ask them in the chat box below our left-hand corner and we can answer them between sessions or at the end. I'm Margie Caller with Foundation. I'm joined here today with Allison Price. And from there we will get going. So John, I'm going to put this into present mode and then you can take it from there. Alright, great. Thanks Margie. So I am the current PTL for the Block Storage project for Cinder. And today I just put together some slides here to go through and talk a little bit about mostly focusing on where we're going for Juno and kind of what our plans are there. And then you know a little bit, touch on a little bit what we did for Icehouse, but that's old news right now, right? So we're all moving on. So anyway, let's go ahead. I'm kind of big on themes for release cycles. So this one I ripped off an old Kinks album. Hopefully none of you remember this. But anyway, the theme for Juno is trying to give the people what they want. So to go to the next slide, what that means is based on feedback that myself and some of the others on the Cinder core team have received from enterprise users and private cloud employers, we wanted to take as much of that information as we could and figure out what we can do in Cinder to make OpenStack more feasible and more attractive for them and make it more appealing for them. The trick with that is trying to actually take feedback from actual users and actual employers as opposed to vendors. In Cinder you have to be careful because it's extremely vendor heavy, which is great. But you have to be careful because their perception of what customers want isn't always what customers actually want. So I tried to distill that as much as I could. And the main things that I came up with in Atlanta after talking with a number of folks was continuing to improve the backups that we have, focus on the compatibility issues in terms of not just backend devices, but also in terms of when you upgrade and go to new versions and things like that, which leads into the next one of not breaking upgrades. The resounding thing was just make sure you keep giving me something that works. So that was good feedback. So the general feedback is it works as it is, keep doing that. Some other interesting things were I need to be able to use my existing gear and then of course quality. And then the one big missing factor was HA. So the next slide is coming up here. I'm going to start touching on some of these individually a little bit. So on the backups topic, of course when you talk about private and enterprise and things like that, whatever you want to call it, one of the most important things that people do is backups. Cinder does have a backup service. We introduced that a couple of releases ago. Works pretty well. So what we do is we do a backup to Object Store. It can be Swift, it can be Seth's Object Store. And there is actually a tape driver in there now as well that you can use. The problem that we have right now though is it's a full backup every time. So there's no concept of real backup management like incrementals and things like that. So that's a little bit of a trick there. The other problem is it's a little bit difficult to restore that volume. So restoring it in terms of getting the data back and getting it on a volume and stuff, that's not necessarily hard. That's easy to do. Of course it works. Getting that integrated back into your OpenStack environment though takes a little bit of cajoling sometimes. So we're trying to look at a couple of things to fix that. So one of the top things is incrementals. So we really need to figure out a way to do incrementals in Juno. There's a couple of proposals out there that are actually pretty promising that came out of the Atlanta Summit. So we're definitely looking at those and I think we're going to make some progress this cycle on that. So hopefully you will see the addition of at least the first iteration of incremental backups supported inside of the Juno release. In addition to that on the restore side, we've got a manage unmanage feature added now which allows you to do things like import a volume into Cinder that was somewhere else. That actually ties in really nicely with the backup piece because depending on how you did your backups or if you use something outside of the Cinder backup service or something like that, you now actually have a method that you can actually pull that volume back in and get it managed again. So, next slide. Compatibility. So this is an interesting one because initially I always thought I knew exactly what people meant. They were just talking about version of version upgrades like Havana to Icehouse, etc. But actually everybody has a different view on this. Some people look at it in terms of the Cinder versioning between the upgrades but almost as many people look at it in terms of features between different back ends. One of the things that I learned in talking to a lot of people is very few people actually only use one back end device in Cinder or even for those folks that do, they have plans or at least they want the ability to have the flexibility to change to something else later down the road. So those are things that I think are really important for us to keep in mind from the Cinder side and I think we're trying to do that and we're driving towards that. Also it's really interesting to see how many people are actually using the reference implementation, the LVM driver, which is really important to me. It's something that I spend a lot of time on and I think a lot of other folks do as well in the community. So the goal here is we want to keep all of the third party drivers, we want to make sure that they're all compatible in terms of the API functionality and stuff like that by having each of them implement their own CI. And what that means is every patch that goes into OpenStack right now goes through a continuous integration test phase. And what that does is it runs an extensive set of functional tests that we have to make sure that all the API calls work and everything's compatible and nothing's broken. And we call that DSVM full. So the idea is if we can get to a stage where we have every driver that's in Cinder actually run that and run that on every patch ideally, then it gives some visibility and some indication to people that, hey, this driver does actually work, it does implement all the API calls, and there is compatibility. So that's the idea there. So as I said, that ensures that all the core API functions are tested and supported on every driver. It's publicly visible. And then of course the other thing is we just keep in mind don't break things between versions. There's not a whole lot of detail I can go into there except for it's just something that every reviewer inside of the Cinder project keeps an eye on. Every time a blueprint comes in we keep an eye on that sort of thing. We keep in mind that, hey, we have to make sure that A, you can upgrade easily and it's not difficult. And then B, it doesn't break anything when you do upgrade. So this is sort of a recap on – I dug into details here, but this is just a summary of those details, right? So we can skip past this one. So on the something that works piece, I had mentioned that one of the things that people have mentioned is keep giving me something that works. So how do you do that? Well, the previous two slides are basically how we do that. So we make sure that every backend driver, every piece of code that goes into Cinder actually implements all the functionality, all the API calls. So the third-party CI testing, that is actually going to be the greatest benefit here and that's going to be the one thing that gives us the best chance of making sure you have something that works. And then in addition, touching on that, the LVM implementation, the reference driver, that's still the primary focus in Cinder. The primary focus is to continue to make that better, to continue to improve that and make that something that's usable and something that's robust for people to actually implement. So one of the other pieces of feedback that I'd mentioned is people want to use their existing gear. It's really common, it seems, for people, especially in the private sector that are getting started with OpenStack, to actually look at repurposing gear that they already have. So in that respect, we still have a lot of vendor participation. We have a lot of drivers coming in. I think there's probably at least another half a dozen or so that are proposed for the general release already. And typically there's more to come. They always show up about halfway through. So I think there'll probably be, you know, probably eight to ten more drivers added. And I think that's a good thing because that gives people choices. And not only do they have choices in terms of what they use, not only do they have choices in terms of, hey, I already have gear from this particular vendor, I can now use it and get started in OpenStack and get going. But also they like the ability to mix and match, as I said before. So these are really important things. And then, you know, the key bullet at the end there, don't lock me in. We hear that all the time in OpenStack. Everybody says they don't want to be locked into a specific solution, right? So those are important things. And I think that we're going to keep going down that path. The key bullet point that we talk about on the reference implementation, and that keeps people from either continuing to use it or using it for all of their deployment or anything like that, is HA. So if you're familiar with the Cinder reference volume service, the way it works is actually you have a single node, so it is a single point of failure. There's no redundancy there or anything else. It's something that we've known as a shortcoming, and we've tried different ideas and stuff like that. In Atlanta, we actually had some folks from Limbit come out in DRBD and have some solutions, and they're willing to try and work with us and try and make something that gives us that HA solution and gives it to us without sacrificing a bunch of performance. So we've got some stuff in the works with them, and I expect to see some really good things in Juno and finally see this problem go away. In addition to that, we also – some folks have come up with a patch that they do on their own in their environment where they actually do some mirroring across multiple Cinder volume nodes using a software raid, and my first response was, hey, I tried that, but the performance is really bad, and they said, ah, yes, but you can tune it XYZ. So we've got some work going there, so the good thing is I think in Juno not only are we going to have HA for the LVM driver, but I think we're going to have choices in which HA solution you want to choose. So choices are always good as long as they all work, right? So next slide. So not only are the things that people are coming back in what we're hearing in terms of what's important to them, not only is that good feedback and stuff like that, but it's also kind of good to point out what we're not hearing, right? So some of the things that I'm not hearing from folks is I'm not hearing that there's a bunch of missing features. I'm not hearing that they're missing API calls or things that they want. I'm not hearing that there's too many options or too many choices in backends, and I'm certainly not hearing that, hey, I don't need block storage, so that's good. And then back to the same thing, you're missing XYZ, so I'm not getting a lot of that. So that's a good thing. That means that we're focusing on the right things, hopefully, and we're going down the right path. So some of the other things that we're working on, I focused on those mostly because those are items that I've received feedback from people that are actually implementing. But some of the other things that we feel are important that we're working on as a team that are primary focuses. The first one is code cleanup and fixes and what I call object sprawl. Over time, Cinder has now been out for a number of releases and we're considered a mature, grown-up project now. There's been a lot of sprawl and a lot of code mess that's kind of developed over the years. So there's an effort in Juneau to actually go through and clean that up. That has a lot of benefits for an end user, mostly because what that means for them is you're going to have something that's actually more robust, easier for us to debug, find issues, maintain, et cetera, et cetera. So it's a really good thing. It's a really important – nobody likes to do it, but it's really important. The other thing is we're working on getting the ability for admins to set up replication. We're trying to avoid doing like a full-blown, hey, this is a replication setup and everything else and make it actually in the API and baked to do everything and anything. But what we want to do is just kind of start by making it easier for an admin to do it. So expose some of the features, expose some of the things for an admin to actually set that up and make it a little bit easier without having to know everything about the backend device. We're also introducing the concept of consistency groups. So those of you who are familiar with databases and do a lot of database work, you know what that's all about. We've actually got some code that's actually been written up in prototypes that we were playing with. We're making good progress there, so we'll see that in June, for sure. The other thing is there's a lot of folks out there that still have devices that utilize the concept of storage pools. So for those folks, there's some problems with that because what happens with the sender scheduler the way it works today is it just goes to the backend and it gets everything, all the information globally from that backend device and it doesn't take into account that some of the pools may be unused or anything like that and do some of the load balancing stuff. So we're going to add the concept of storage pools so that that can be broken out. It's going to be invisible to the end user still, of course, but what it's going to do is it's going to make the scheduler do things more intelligently. So that's a good thing. And then the other thing that's been kicked around for a really long time and I think it's finally going to happen in Juneau is scheduling local storage on compute nodes. And what that means is there's use cases where people actually want to use local attached storage, local attached disks that they have in their compute node as a sender volume that they attach to their instance. Currently we have no way of really doing that without actually running a full-blown sender volume service on every compute node which in my opinion is kind of a step backwards from where we've been going. We've got some ideas. Vishaya had some ideas back in Portland that we just never actually made happen. But I think we've got between myself and some other folks that are really interested in working on this. There's also a lot of demand. I think in Juneau we're actually going to make some progress and we'll get this feature out. So look for that. That's going to be something that's really cool. So in terms of what we're missing, things like that, I just wanted to put this slide in here for folks that see it. If you're deploying a private cloud with OpenStack or you're thinking about or considering it, you've researched it or anything like that, let me know what we're missing. If there's things that you see, if there's gaps, send me an email, grab me an IRC, whatever. But let us know, let the sender team know what problems you might have, how we can make it easier and more beneficial and make it advantageous for you and help you get started with using OpenStack or if you're already started or whatever, how we can make it better for you. So we're always open to feedback. We don't get enough. In my opinion, I think we can always use more. And then finally, the other thing that I always tell people is get involved. Continuity with OpenStack is not just for software developers. Open source is about more than just writing code. So there's documentation, there's filing bugs, feature requests, and then the most important thing is just feedback. Just giving feedback to the community, to the developers, is a great way to contribute to OpenStack. That way, we're actually building something that you want. So it's extremely important. And then finally, the other thing is support others. If you've deployed OpenStack and got things running and have lessons learned and stories to tell and stuff like that, share those stories and help other people that are starting on this journey as well. So with that, that's all I have. So I'm not sure if we want to take questions now or later. I think we can do later. I think we can do later unless people have something they want to ask right now. If anyone has any questions for John, you can put it in the chat box. And we can answer them later if you don't have any right now. Okay. I don't see any right now. So thank you so much, John. Thank you, Arwin. Thank you. Okay. This is Margie again. Okay. Well, I am going to turn it over now to Owen Glenn. And Owen, I'm going to put your presentation mode. So Owen, are you out there? I am indeed, Margie. And thank you very much for the intro. Sure. Go right ahead. Cool. So hi, folks. My name is Owen Glenn. As Margie said, I'm a PTL of the Tenementary Program at OpenStack. And the main constituent project in that program is CELOMETER. And that's probably the codename that you're more familiar with. So one thing I wanted to kind of cover initially was this idea of this kind of persistent identity crisis around CELOMETER. So since the project was mooted for the absolute outset many cycles ago, there was kind of confusion about the mission of the project and the extent to which it was limited to just doing metering in the sense of capturing data about the usage of user-visible cloud resources and generally feeding these data into consumers such as billing engines or whether that's kind of a wider scope to its mission and included measurements of all kinds, including monitoring, for example. So after many kind of debates around this and some of them circular and a kind of Edmunds load over multiple cycles, we finally come to the point where we've tied down an explicit mission statement and agreed that with the technical committee, which are the, I suppose, the main kind of steering group from a technical oversight point of view in the OpenStack community. And we wordsmithed a agreed mission statement and copied it into this slide here and put a bit of emphasis on some key words. And this is like literally hot off the presses from the governance repo. It's a patch that I had kind of an extended review over many weeks, and it landed a couple of weeks ago. So this basically represents the state of the art as far as the definition of the salometer project is concerned. So basically what we're talking about is a mission to reliably collect measurements. So reliability is key there, and we've reflected that in the mission statement. And the idea is to basically collect measurements around both the usage of cloud resources, and this is the type of information that could potentially feed, say for example, a billing engine, but also the performance of those resources. So you can imagine if you tie it down to say an instance, an actual virtual machine in an OpenStack cloud, the usage of the resource would relate to how long the instance is actually running for, when it was booted up, how long it ran for, what flavor it had, and so on, and when it actually stopped existing and was terminated. Whereas the performance aspects of it would include things like, well, how much memory was it using at a particular point in time, or what was the CPU utilization percentage at another moment in time. So our mission is to reliably collect both types of utilization data as they relate to virtual resources. So these will be things like instances, as I just mentioned, or images in glance, or volumes of block storage, for example, in Cinder. And also to physical resources that map directly onto the fabric of the data center. So we're talking about host level information, for example, say the CPU load averages over an entire host, as opposed to just the utilization associated with an individual instance. And to gather all of that data, right, so that's our first name, and basically to slurp it all in via a pipeline, such that these data are persisted, so that they're stored reliably in some kind of storage engine. And we've got a variety of pluggable storage engines in Solometer, ranging from no SQL data stores such as MongoDB, which is probably our canonical storage driver, and it's the one that most distributions, open-stack distributions, will actually support for use in production. But we've also got storage drivers, for example, based on HBase, and DB2, and we also have even a SQL alchemy storage driver. That's more suited to kind of small deployments or outputs with concepts. And the idea is that once these data are persisted in a metering store, they become available for later retrieval. And that retrieval is affected via a RESTful API, as per the Common Practice and OpenStack. And then the analytics provided via that API are at a relatively simple level. So the idea is that Solometer itself is not in the business of providing complex analytics. And we do provide basic aggregate functions, average standard deviation, minimum, maximum, that kind of thing. But more complex analytics are assumed to occur outside of the scope of Solometer and be applied to the data retrieved via our public API. And then the last kind of clause of this mission statement relates to the Solometer alarming feature. And the idea here is that we have the ability to do and to watch trends in certain statistics and to trigger basic actions, such as calling out the web hooks when some defined criteria are met, which is usually expressed in terms of a threshold for a particular statistic over a particular time window having been crossed. And this is the aspect of Solometer that's used in conjunction with HEAT to drive auto-scaling actions. This is basically the mechanism that allows you to define a set of instances as an auto-scaling group. And these would be defined in your HEAT template. So HEAT is the orchestration engine in OpenStack. And to basically associate high and low water marks in terms of the busyness of those instances. For example, to say, if the average instance across my auto-scaling group is running relatively hot, as in the average CPU utilization exceeds, say, 70%, well, then we want to share the load a bit wider. So we want to spin up more instances and have that happen in an automated fashion. And conversely, we may have a low water mark where you say, well, if the average instance is kind of running idle at this point in time, it's an indication that we're overscaled. So we want to basically shut down some of the instances that make up our auto-scaling group and share the load or concentrate the load over a narrower group of horizontally scaled-edge web servers or mail servers or database servers or whatever your instance happens to be hosting. So that's a fairly wide kind of range of things that Solometer does. And hopefully the fact that we've agreed now this mission statement with the technical committee, and basically it's all the eyes are dotted and the teeth are crossed as far as the governance repose concerns, that that will kind of put to bed the debate around the scope of Solometer and whether any particular new feature could be characterized as mission creep or whether it falls within the well-understood scope of this project. So that's slide two. Moving on to slide three, and I've just kind of set the scene, I guess, there with talking about our mission statement. So I think it would be worth talking a little bit about where we are if we were to take the current state of play and how would you define the health of the project at the outset of, say, the Juno development cycle? Well, as luck will have it, we had a very useful and healthy checkpoint recently in the form of a technical committee gap analysis. So moving on to slide four, let me explain what the background to this was first. So heat and Solometer actually graduated after about the same time, right before the outset of the Havana development cycle. So there was a couple of cycles back in time. And at that point we were the first project outside of the traditional group of five or six that formed the original core of OpenStack. So in a sense, the rules were being kind of just being defined at the point in time at which heat and Solometer graduated. And as time has progressed, more and more projects have appeared in the OpenStack ecosystem and there's been quite an explosion of activity, I guess. And the kind of the standards and the rules applied as far as graduating from an incubating project or an integrated project have become more and more defined. There's been kind of a gradual inflation, I guess, in terms of the standards required and a raising of the bar, which I think is a healthy thing. But in fairness, it also meant that the projects that graduated first got an easier ride as it were. So the technical committee in the interest of fairness decided to do a retrospective gap analysis on all of the existing integrated projects and decide whether those projects met the new higher bar that is currently being applied to projects that have requested integrated status in more recent times. That would be projects such as, say, Designate the DNS as a Service or Marcoli, the Dequeuing Service to take two relatively recent examples. So I think this is a very healthy thing and it provides a very kind of useful checkpoint to see where a kilometer was and where it's at now and whether it meets that higher bar. So the outcome of that gap analysis were that a number of concerns were identified, a number of gaps were identified by the technical committee. And this has been the pattern when these gap analyses have been applied to each of the projects in turn. And then the idea is that basically the projects commit to resolving these gaps, resolving these concerns in pretty short order. So we've committed with the technical committee to show front-loaded progress on these gaps and within the scope of the Juno-1 milestone, which is just passed, and completing this work for Juno-2, which is due in late July, I think it's July 24th. So in addition to the mission statement that I mentioned earlier, the actual gaps identified a list of them out here, one of them was to have a viable sequelscene driver. This is the storage engine for Solometer that stores the metering data in a relational database accessed using the C-Clockby library, which is a Python and Monogic relational mapping toolkit. And we had a C-Clockby driver, but it suffered from some performance issues. It was essentially there as kind of an alternative to Mongo, but most of our development effort had gone historically into MongoDB and that was the most futureful and most widely used and certainly had, and by far, the most superior performance characteristics of any of our storage driver. So the C-Clockby one was kind of the poor cousin and its usage within the continuous integration gate had introduced some issues. Now I suppose we should kind of press all remarks associated with C-Clockby driver with a caveat that is not really recommended for production use on any kind of large scale because relational databases are not really well suited to storing metric data. That's just kind of a fact of life. But it's something that's useful to have for running in situations where Mongo is not available, which would be the case in the continuous integration gate because the version of Ubuntu on which most of our continuous integration runs, that's 1204 Ubuntu precise, had only an older version of Mongo than our salamity requirements. So basically we've done a lot of work on improving the performance of the C-Clockby driver such that it now is acceptable and as a viable citizen within the continuous integration gate and it's suitable for relatively small proofs of concept and small deployments if people wish to use it. That's been quite an interesting technical exercise because it's involved quite a lot of close interaction with the author of SQL Anthony, a guy called Mike Baer, a colleague of mine in Red Hat. And yeah, I think we brought that from a stage where its performance was unacceptable to a much healthier state within the scope of the June 01 timeframe. Now a very related thing was Tempest. So Tempest is an integration test framework within OpenStack. And this is basically where we, obviously we do unit tests per project but we also want to be capable of testing things with various services used together and everything spun up in a realistic way instead of external dependencies being mocked out. And Tempest basically is the harness that gives us that capability. Now with a lot of Tempest coverage for teleometer that was written during the previous release cycle, the Icehouse release cycle, but we didn't have the capability to actually switch that on within the gate because of the performance issues with the C-Clockby driver. Now that those have been resolved we've landed a wider variety of Tempest test coverage and we've also utilized our own capabilities API which allows branchless Tempest to run against both the latest on Juno and also the stable Icehouse branch which wouldn't have all of the performance improvements to C-Clockby. And the Tempest test is capable of dynamically discovering whether or not the version of teleometer that's been tested really has the performance characteristics that allow various test scenarios to be unskipped. So that's all kind of very neat and is working well. We also want to enable testing against MongoDB in the integration test because clearly that's our preferred and most widely supported and widely used storage driver. And this will probably involve us either taking advantage of the continuous integration switch over to Ubuntu trustee using Fedora 20 and or CentOS 7 in the gate to allow us to access the more up-to-date version of Mongo that teleometer requires. The third item there in the list relates to documentation. So we have quite a lot of developer oriented documentation which would describe things like how do you spin up a DevStack and run this thing in a sense of a code contributor might view it. But we didn't have as much an invitation that related to the point of view of a typical user or even a typical operator. So we're definitely addressing that in conjunction with the documentation team. One thing we've done is we've taken a kind of getting started guide that I wrote for the RDO website and we're taking that and basically recasting it in a kind of a distribution agnostic form. And we're also writing a new operator oriented guide. And lastly we've got the grenade upgrade testing harness. So this is another part of the quality assurance pipeline within the OpenStack community which is very useful. Which basically involves testing the resilience of various services across upgrades. And it's interesting because it's the type of thing that in the early days of OpenStack problems were really only discovered in the field when people actually tried to upgrade from release A to release B and problems that might have been anticipated or not encountered in testing occurred in production which obviously was badness. So the grenade testing framework is a very simple idea. It just installs the old version of OpenStack, spins up some resources, shuts down the old, leaves the resources running and moves forward to the newer version of OpenStack, reruns all of the services and then does various assertions around the survivability of those resources. So individual projects can then participate in grenade by adding their own specific assertions. So in Solometer's case it would be, for example, that a statistics query which spans the old cloud via the upgrade downtime into the new cloud that basically such a query returns valid data and a mixture of the new and the old metric data is valid and doesn't cause things to explode and everything is good in that sense. So they're basically good actions that we've been mandated by the technical committee. Moving on to slide 5 though, let's get into the kind of the meteor more interesting stuff. So a lot of that and those mandated actions from the TC were around kind of being a good citizen within the continuous integration framework or providing documentation, those kind of good citizenry type requirements. But probably of more interest to a lot of people on the call would be what's coming up in terms of features. And I guess the way I'm viewing it at PTL is we've a strategy over the genocycle to basically have two parallel efforts in play. One of which has got a kind of a longer time horizon. It's kind of a multi-cycle effort. And it's around kind of paying down the architectural debt that we've kind of accreted over time and basically changing the nature of our measuring store. Then the second parallel effort is more that shorter time horizon and it's more featured on incremental improvements to what we already have. Addition of new features that are specific interest to a lot of current users and general improvements of where we are now. So let's talk about the longer time horizon and the more forward looking thing. So this is something we spent a lot of our time in Atlanta at so much discussing. And this is basically the idea of reimagining Solometer as more of a time series data as a service service, as opposed to a service that stores data which mixes in actual metric data points with snapshots of resource metadata. So if you were, for example, to take the current measuring store in Solometer, for example at an installation that was based on MongoDB and you're to fire up the Mongo client and have a look at what's in our meters collection. You'll see that for every data point that we store, we'll say the CPU utilization of a particular instance is 42 at this moment in time and then a minute later it's 40 and then two minutes later it's 41, whatever. So that's a trend in terms of some numerical value. But associated with each of those data points, we also store a snapshot of the resource metadata. Now this gives us a lot of clarity in terms of allowing us to query and slice and dice these data in a lot of novel different ways, right? But it also means that our measuring store is quite heavyweight and not as efficient as it could be in terms of either space or time because these metering or these snapshots of the resource metadata are quite kind of beefy things and most of these data actually don't change very much. They're completely static or infrequently changing. So for example, the flavor of a resource, or flavor of an instance rather, might only change when you resize that instance, which is a relatively infrequent event that may not even occur for lifetimes of many, many practical instances. So what we're doing with time series data as a service is we're splitting off the resource representation from the pure metric data points. And then we're using the ability to persist individual events, which are basically notifications of some life cycle change in the history or the lifetime of a resource to basically reconstruct the resource state timeline if necessary. But in terms of and the actual trend recording, what we want to do is store these data in a much more lightweight way and shorn of the more heavyweight resource metadata snapshots. And that allows us a lot of flexibility as to how we store these data and a lot more efficiency in terms of the storage footprint and the time characteristics of querying and so on. So we've prototyped the implementation of this in Stackforge. We spun up a new project on Stackforge called Nachi. Just kind of, if you look at the code you'll see there's kind of an Italian food team pervading through it. And that's being driven by Julianne Donchou who's my predecessor at PGL of Sonometer, who served as PGL for two cycles. And the idea here is that it'll contain a canonical storage driver and an analytics engine that's based on the popular Pandas library in Python and Swift and providing the actual storage. And then we also have the capability to plug in alternative storage drivers. And the idea here is that we will build integrations with state-of-the-art in terms of metric-oriented databases. For example, we've got a good conversation going at the moment with the folks behind InfluxDB which is a relatively new metrics-oriented database implemented in the Go programming language. That's very interesting. We're going to have the main contributor to that coming to our mid-cycle in Paris next week. And we also are looking at some other metrics-oriented databases like OpenTSDV. But we'll also have our canonical driver that has no external dependencies in terms of external services and that'll be based on Pandas and Swift. And the idea here is that we'll develop this over multiple development cycles. We'll have a prototype implementation ready by the end of June and basically this will form the basis of the next major iteration of the Selometer API which will be the V3 API. But for now the V2 API is where it's at. That's what we're supporting and there'll be a long period and a long deprecation path as is usual in OpenStack of multiple cycles. So there's no need to be concerned that the current API is going away or anything of that ilk. So that's the kind of forward-looking stuff. The more immediate goals involve incremental improvements, various features that are widely requested and much desired and are kind of of particular interest to individual contributors in the Selometer community. So just jumping through these and one is the over the eyesight cycle which is the previous development cycle in OpenStack. We added the ability to do SNMP monitoring of host metrics. So for example you can acquire things like the CPU load averages at a host level. But the way SNMP is actually structured is there's a bunch of different metrics each identified by an OID that basically could be gathered and every installation really I guess has to choose which particular metrics it's interested in. Currently the way it's implemented in Selometer is for each new metric, each new OID that we're interested in at an SNMP level we have to write a bit of Python code to basically grab that value and massage it and distill it and produce a sample that's then submitted to the metering store. So the idea is to do this in a much more declarative fashion that's completely driven by CONSAVEN. It doesn't require any new code. So you can give it a code-free metering and it allows basically people to just pick and choose from the menu of options. For example these could be SNMP OIDs or it could be metrics that are available via some other source, say for example via the vSphere API if you're using VMware. And in a completely declarative fashion code-free you don't need to write any new polling code basically enable the metering and define the mapping from this particular external metric to the native salameter representation of that sample. Another interesting feature that we're adding is event storage. So basically we have this implemented in our SQL Alchemy driver already and this was contributed by some of the folks behind the StackTack framework. And basically what we wanted to do is extend that so that the same functionality is supported in a variety of different storage drivers in particular HBase and MongoDB. And what we intend to use these events for is basically these are notifications that tell us that something significant has happened in the lifecycle of a resource. For example an attempt to migrate an instance has started or an instance resize has finished. And these basically allow us to reconstruct the timeline of the resource state and after the fact without necessarily relying on continual snapshotting of the resource metadata as we did previously. Another kind of technical depth that we need to pay is around the scalability of the central agent. So the central agent basically is a component of Solometer that does things like drive the SNMP polling it also allows us to retrieve data from public RESTful APIs exposed by the various services. The thing is that the way the central agent was originally written there's no way of doing load sharing. There's no way of starting up two instances of the central agent and saying to one, okay you take care of this and the other one you take care of that and basically allocating a disjoint load to each of them so they didn't kind of duplicate their efforts and step on each other's toes. So the result is basically the central agent is kind of like a single point of failure. I mean you can do it you can run a multiple search for central agents in kind of active passive mode but you can't scale it out in active active mode which is the way you want to do it generally. So what we're intending to do is to use the task flow library which is widely used in OpenStack and to use this as kind of a work partitioning engine to allow us to start up multiple central agents and for them to share the load between them and coordinate their efforts. And another interesting feature is around IPMI sensor data. So IPMI provides the capability to get a lot fairly low level data things like very hardware oriented things like temperatures and time speeds voltages all that kind of hardwarey stuff which is all goodness and all the interest sometimes but the thing is about IPMI well there's two issues with Solometer invoking on it directly. Number one there's a credentialing issue so generally IPMI is username and password and we don't want to leak these credentials into Solometer and we didn't really want to take the dependency on something like Barbican yet because it's kind of a bit early in that project slide cycle. So Ironic already has the capability to own these credentials then there's also the issue that over polling of IPMI as an over frequent polling can cause issues I mean there are known issues around and lock up being caused by just calling out IPMI with too tight a polling cycle. So it seems logical for Ironic to remain the owner of both the credentials protecting the IPMI interface and also to control the cadence or the frequency of that polling interval. So the way we agreed with the Ironic folks at some of what's just gone is that they will control the polling cycle and they'll emit an amputee notification containing a dump of the sensor data from IPMI and then Solometer will take that, massage it, filter it, take the data that's kind of varying and interesting and according to our configuration for particular deployment persists some of these data in the form of Solometer measures. And then the last thing I mentioned there in terms of our incremental improvements is a variety of integrations with the networking services provided by Neutron. So for Juno 1 we've already got the capability to acquire statistics related to load balancing as a service. So you can kind of see how many connections are open at a particular time and you know what the state of the health probes are, that kind of thing. But going forward to Juno and we've got a contributor from Cisco who's very interested in extending this to do similar things for the firewall as a service and the virtual private network as a service and all of the other star and AAF is provided by Neutron so as to provide a more complete view of what's going on. So basically that's the incremental improvements. So yeah moving on to slide 9 I guess that's kind of it as far as what I want to talk about today. So I guess thank you for your attention folks. I suppose we may have a couple of minutes for questions. I think I've probably gone over time. Dependency to do that when I get into the zone but then I'll hand it back to Margie at this point. Great. Thank you Owen. Quite all right. That was wonderful. Let's see let me get back over to the portal. Does anyone have any questions they want to ask today? I know we have some people on the line. Do you want to I actually did not mute the line so you can go ahead and ask them out loud or you could put them in the chat box or you could not ask any. That's okay. Sometimes people ask questions after the webinar when the meeting burner meeting is complete you'll get I think a survey or something and you can send those questions back over to us too. So that works fine as well. Anyone else questions? Okay. Thank you very much for your time. I know you're both very busy. For those on the call this recording will be on YouTube probably by the end of the week early next along with the rest of the PTL webinar series that will complete I think mid next week before July 4 is holiday here in the States and with that I will end the webinar and thanks again to John and Owen and have a great day. Thanks. Bye-bye.