 Okay, welcome to the PTL overview series from Juno to Kilo. We started this series so that we can do an extension from the summit where the PTL shared our project updates. Today we have Owen Glenn, David Lyle and John Dickinson sharing their project updates. And we're going to go ahead and start with Owen Glenn who will provide updates on telemetry. So Owen if you're ready? Thanks Allison. I'll take it away. Basically I'm going to provide, I'm the salameter PTL as Allison said, and I'm going to provide an update on what we're planning to do over the Kilo release cycle. So next slide please. We'll start with just a little reminder. And on slide two I've got the mission statement that we agreed with the technical committee over the previous cycle. And this provides, you know, a fairly terse kind of summary of what we're all about in salameter. So I've kind of divided it up there in the slide and kind of the three phases of what we do. So the purpose really is to surface insight into what's going on in your cloud. In order to do that we need to collect measurements basically of how the physical and virtual resources that comprise your deployed cloud and how these are being used, how they're performing, who's actually using them. And then these data once collected obviously, you know, in order for them to be put to use later on down the line. We need to persist these data so that it can be subsequently retrieved and analyzed. And also we need to be able to trigger actions when some defined criteria are met. And a really good example of that would be heat auto scaling. So basically heat auto scaling is a mechanism whereby the content or the membership of a group of instance can be dynamically adjusted according to the trend in usage that's observed of those instances. And that's an example of where an action has been triggered by some data collection that salameter is doing. And then the actual triggering of the action is basically driven by a feature of salameter called alarming. So moving on to slide three, that's basically our mission. And the next thing really is how are we applying this mission over the period of the Gino release cycle? Well here are the things on slide three. I've given it kind of a high level laundry list of the things that we spent our time in Paris talking about when we weren't doing touristic stuff, we weren't traveling around looking at the Eiffel Tower much. Most of it was spent in deep conversation in Conclave where we discussed basically our plans for the upcoming Gino release cycle and came out with the set of prioritized teams that we're going to attack over the next six months. So the highest priority thing and the thing that we're really focusing on is completion of this new time series data as a service, the no-key project we're calling it, and the migration of salameter to use that as its metric storage layer. So that's really going to consume a lot of our attention. And basically it's been a multi-cycle effort. I'm going to talk in detail as to what it comprises, why we're doing it, what the status is, and what the remaining work looks like over Kilo. But in addition to that, we've got other parallel efforts going on. And I'm going to provide a little bit of detail on each of these on this update. Now one thing that's quite important that's happening across the board over Kilo in OpenStack is the idea of taking a lot of test coverage that was previously provided by Tempest which is an integration and test suite that spins up a lot of services and tests a lot of interactions between those services, moving a lot of that coverage so that it's now within the individual trees of the individual services with some other plan testing improvements as well that fall under that category of providing the right kind of test coverage. And I'll give a bit more detail on that in a few minutes. One thing we want to do is to basically make the segregation between tenants and the role-based access control and effect to make that configuration richer and more flexible in Solometer. We want to reach a point in time at which the notifications that the other services emit, services such as Nova, Glance, Neutron, Keystone, etc. Those notifications are currently consumed by Solometer. And we want to basically allow that notification-based interaction to become as contract-based and as stable as a true API. And that will require some work both on the consumer side, in consumers such as Solometer and also related projects such as Stacked Act, but also a lot of change on the emitter side. So that's a very significant piece of work. And it's something that's been mooted many times. It's been kind of oft-disgusted but never actually delivered upon. So what we're really trying to do over Kilo is finally put that issue to bed and finally basically promote notification-based interactions so that they're kind of like first-class citizens in the open-spark world. We have a number of improvements on our roster around deployment flexibility. So ways in which the complexity of actually deploying a Solometer, the set of Solometer agents over a non-trivial deployment of OpenStack, not just a single node POC or something small like that, how that flexibility can be improved and the complexity reduced. We also have a few improvements around how we process events. So currently Solometer basically has two different ways on which data can be acquired. One is kind of an act of going out and grabbing that data, a kind of a polling model. And the other is more passive, kind of sitting back and allowing the services to tell us what's going on via notifications that are delivered over the Osno messaging bus. And we've got a number of ways of improving how we actually process those notifications. And lastly, I've got a line item that's kind of pretty much assigned to me and that's a collaboration with a very interesting and related project in the open-spark ecosystem called Manasca. So Manasca are all about monitoring at a very large scale. It's a project that grew out of an effort within HP and now involves contributions from several different large OpenStack companies including IBM and Rackspace. And there's somewhat complementary to what Solometer does, but also there's some commonality there as well. And we're going to explore how we can kind of collaborate with the Manasca folks and where our interests align basically, achieve things together. So let's drill down in a bit more detail into each of these themes. So firstly, this time series data as a service, this no key project, what's it all about? So first to recap, this is actually very similar to a slide that I used in the PTL update for Juno. And basically I'm just recapping what no key is all about. So the goal here is to provide efficient metric storage. So you may well ask the question, well, Solometer is obviously involved in storing metric currently. Is that inherently inefficient? Well, there's a couple of kind of drawbacks to the approach that Solometer currently takes. And one of those is the fact that we snapshot resource metadata alongside each individual data point that we store. Now that provides an awful lot of flexibility and it also provides a very good record of the evolution of the resource state, the timeline of that evolution. But the downside is that much of this metadata is either static or very rarely changing. So there's much more efficient ways of storing this near static information than continually snapshotting it alongside each individual data point. And the other thing that we want to change, we want to kind of upend the approach is classic Solometer to coin a phrase is predicated on this idea of all aggregation being done on demand. If you issue a statistics query and you have a certain ad bucketization that you want to, for example, per hour, that aggregation is done on demand, on the fly, for each individual query. And if the same query is omitted a second later, it's done again. So what we want to do is shift to a model whereby we eagerly pre-aggregate and roll up these data as they're being ingested. So those are the key differences really between how we view this time series data as the service playing out. It's been implemented initially at arms length from Solometer via a project that was spun up on Stackforge by my predecessor, Julien Bonjou, my predecessor as PTL of the Solometer project. And it was envisaged from the get-go as kind of a multi-cycle effort that we would play out over Juno and Kilo. And then we get to a point at the conclusion of Kilo whereby we were ready to migrate a Solometer core to using Loki as its primary metric storage layer. So currently there is a canonical analytics and storage engine provided by Loki which is based on the Pandas Analytics Library. It's a commonly used data analytics library in Python and using Swift as the storage backend. But our intent is to also provide alternative storage drivers so we follow the standard OpenSack pluggable model. And the idea is that we'll have alternative drivers based on other specialized metrics-oriented databases such as InfluxDB and OpenTSDB. Those are the two that we're currently actively working on. So that's kind of a recap of where Loki came from. So where is it at currently? What's the status? So moving on to slide 5. I've listed there basically a kind of a laundry list of what we've achieved so far and what's currently in-flight as far as fairly coarse-grain functional areas of Loki are concerned. So completed we've got the core metrics and resources API. So it's a REST API in the typical OpenSack style. And basically it's going to form the kernel of the V3 Solometer API. We also have a single storage driver that's completely built out that's based on Pandas and Swift as I said. We've also got the aggregation model and a policy-based model that determines what level of aggregation and how to roll up occurs for each individual metric. It's a fairly rich model because you can attach different policies to different metrics and have different levels of expiry and time to live related to different metrics. We've got a dispatch mechanism in place within the Solometer Collector service and this is a API-based model. And by that I mean that the dispatcher uses the REST API to push the metric data points up to the time series data as a service API. And the intent basically was for that to be the kind of proof of concept, the proof that the Solometer data could be stored and retrieved from this new No-Key service. But as I'll mention later, we've got an alternative more efficient dispatch model in mind where the No-Key storage driver effectively runs in process within the Solometer Collector service. And we also have the Keystone integration piece all gone. So the typical kind of call out to the authentication layer and also the out-of-the-box, row-based access control policies. Now in progress we've got our two alternative storage drivers. One is based on OpenTSDB which is a moderately widely used metrics-oriented database that's based on page-based. So it's a type of thing that one would run over an adobe cluster. And then also in FluxDB which is a very interesting database that's implemented in Golang that, you know, it's again specialized for metric storage and provides native features around down-sampling of data points and is highly optimized for this particular type of storage problem. We've got some logic around custom aggregation whereby you can have pluggable components that basically provide additional aggregation logic. So clearly we're going to have our basic aggregation functions defined, things like maximum, minimum, averages and sums and counts, and even more exotic things like standard deviations. But sometimes you want to go even further than that. So custom aggregation layer in progress being implemented and a number of an example applications of this including moving averages and also host, winters and forecasting. Another thing that's in progress that's quite interesting and will be crucial for our alarming integration is the ability to do cross-entity aggregation. So by that I mean the ability to aggregate data points that have originated from different resources. So you can say either give me the CPU utilization for this individual instance or else give me the average CPU utilization over a group of related instances. For example, the instances that make up an auto-scaling group as defined by heat. And lastly there we've got, which I mentioned earlier, we've got in progress an effort towards providing an alternative dispatch model from the salameter collector service that's basically in process as opposed to API-based. So in that case the collector would load the Nokia Storage Driver and call out directly to InflexDB or Pandas Plus Swift or OpenTSDB or whatever it is as opposed to making a direct invocation on the REST API. So that's kind of where we're at as far as Nokia is concerned. So what additional work have we planned for Kilo? So basically the crucial things that need to be achieved before we can declare a victory on this, we need to recast the V2 query API which allows you to basically slice and dice salameter data in many different ways. We need to recast that over Dohe semantics so that it takes account of the restrictions that would be imposed by this model of not snapshotting resource metadata for each individual sample data point. And remember, having put in that restriction in place is key to making the salameter metric storage strategy, making that massively scalable. We also need to rebase the salameter alarm evaluation logic on the V3 metrics API. So this will be effectively the retrieval API that Nokia supports. We need to provide logic to allow customers to take pre-existing measuring stores. So a customer for example, or a user who basically had a deployment of salameter running for some time and was using the classic salameter storage layer. We need some way of capturing that data and basically extracting it and distilling it and basically presenting it up then as equivalent data to the way Nokia actually stores in pre-aggregated rolled-up form. And lastly, in order to, I suppose, make clear the benefits of using this alternative storage strategy, we've already done quite a lot of profiling with a lot of potential optimization that we're going to attack over Kilo. And finally, we're going to publish all of these results and indicate the potential benefits so that people are very sure before they switch from classic salameter to using the Nokia storage layer as an alternative. Now of course, the classic salameter way of doing things will be maintained on a deprecation path and the standard in OpenStack will be for something like this to maintain for at least two cycles going forward. So it's not something that we're going to pull the rug out from under existing salameter users. There will be a long transition phase and basically people will be given plenty of notice. But of course, it's always good to have a very solid case for early adopters to move forward and that's what we intend this detailed performance profiling and effort to provide. So in addition to that, we've got a bunch of other parallel efforts that are unrelated to the Nokia project. I spoke about these in fairly broad terms. I'll give a little bit more detail now. So in terms of testing, what do we want to move to? Well, we want to follow the trend that's happening right across OpenStack of moving a lot of our coverage out from this kind of global tempest test suite into functional tests that are more directly associated with our project repo. So we're calling these injury functional tests in the sense that they live alongside the actual code that's been tested in the same Git repository. This will allow us basically to not be as restricted in terms of how these integration tests are actually constructed. So a good tempest test is a pure black box test. It doesn't assume anything about the underpinning implementation. Whereas we want to be able to test things in more of a gray box method. We want to be able to make assumptions about the internal implementation. We want to be able to write tests that do things like accelerate the metrics gathering cadence that Selometer uses so that the test can complete in a reasonable time. So we want to, for example, change some configuration option that causes some Selometer agent to gather data points at a much faster rate than it normally does and then make some kind of assertions around the metric data points that have been gathered and do that all within a reasonable time. We also want to basically produce a set of API tests that are much more declarative in form that basically provide kind of dual duty such that you can read the tests that are unencumbered by a lot of complex Python code. And instead, as well as providing test coverage, provide an almost organic form of documentation of how the API is intended to actually be used and how the form of requests and responses but in a very kind of high level and digestible form. And lastly, we want to use the rally test framework to basically introduce scenario tests. So these are tests that basically induce some workload over Selometer and then basically measure the results in effect. And that will basically allow us to measure performance improvements in a very convenient way and also allow us to basically watch out for performance degradations where they occur and to catch those early. Another area that we're going to dig into is this whole configuration of segregation between tenants. So up to now, there's been kind of an all-or-nothing model in Selometer between basically users who have the admin role and users who are non-admin. So admin users can see all. They're omniscient. Basically, everything is visible to them as far as Selometer is concerned. Whereas normal users can only see the data and can only reason over that data and alarm on that data associated with the resources that they themselves own. So that's kind of, I mean, it's useful in terms of tenant segregation but it's very kind of absolute. Whereas we want to have a much more kind of nuanced model and use the role-based access control mechanism that OpenStack provides. And basically to start leveraging the more forward-looking features of Keystone including this idea of domains. So this notion that the administrative role isn't something necessarily global but it's something that can be partitioned between different related groups of users via this notion of domains. With a number of different improvements around deployment flexibility, so we're going to merge our very simple, our very similar central and compute agents set up with a single polling agent that can be run in a similar mode to the current central agent or compute agent or can do dual duty. And that will make small deployments more simple. We want to basically be able to centralize the storage of the pipeline configuration which is currently stored in a flat file that needs to be rolled out to each individual node. The pipeline that YAML, the existing Kilometer users will know and love. And we want to basically allow that to be centrally stored so that it can be changed in a more global fashion across large deployments. And we also want to, one thing, another thing that will make deployment simpler is this idea of allowing the metrics scattering over SNMP to be truly the characters to be driven by config so that you can decide which SNMP metrics you're interested in and change those on the fly very easily. Smaller parallel efforts, we want notifications that are emitted by services such as NOVA, Neutron, Glance, etc. to be, I mean currently these are essentially just preformed dictionaries. There's no contract involved, there's no stability. So we want to go ahead and schematize all of those and we're doing that as a joint effort with some folks from the Stack Stack project which is a related project in the open stack ecosystem also alongside Selometer, probably the primary consumer of notifications. We're going to improve our events pipeline so that the events database is completely split off from the metrics database so you can decide to store your events, for example, in Mongo and store your data points, for example, in Nokia or in HBase or wherever, you know, any of the other storage drivers that are supported by Selometer. And we're also going to use the same model of coordination between scaled-out notification agents. This is the agent in Selometer responsible for consuming events as we currently use for the central agent and the compute agent where we want to provide a kind of a mutually exclusive partitioning between individual agents such that they don't step on each other's toes and they don't duplicate work but yet no work also falls through the cracks and that can be done in a way that takes account of the fact that the pool of currently running agents can change over time. And last we've got a couple of line items around collaboration with the Related Monastka project. So I'm currently working on figuring out if we can reuse or leverage somehow their anomaly detection engine which is a very interesting component in Monastka that we don't have a direct analog off in Selometer. We both have a common concern around InfluxDB. They're using InfluxDB as one of their storage options and we also intend to do so for NoKeep so with common concerns, for example, around getting Influx into the continuous integration gate upstream. And lastly, Monastka are using Apache Kafka as their kind of high throughput internal messaging bus and that's something that Selometer could learn from that we use also messaging both for external and internal messaging and basically Apache Kafka provides several kind of benefits around and throughput and scale etc. So just my final slide here is just emphasizing the point that we're available for anyone who has any further questions or wants to kind of any more depth on any of these topics to reach out either on IRC, on our HashOpenSpecSelometer channel or you can ping me directly over IRC on Eaglin at FreeNote and my home time zone is GMT so take that into account if you're wanting to chat over IRC. We have our weekly meeting at 1500 UTC on a Thursday so you can join that if you want to go deeper with any of these discussions and of course we've got the OpenSpecDev mailing list. So that kind of concludes my update for Selometer so I'll hand the conch back to Allison at this point. Fantastic, thank you Owen for providing those updates and I'll also include his contact information in the description below so again feel free to reach out to him at any time if you do have any questions. Alright, next we have David Lyle on the line who will provide updates on Horizon. Great, thank you Allison. So let's just start with the Horizon QLO update on David Lyle on the OpenSpec dashboard PTO. I just want to start off with even the next slide. So I just want to start out like a lot of these reminding what our mission statement is in Horizon basically to provide an extensible unified web-based user interface for all integrated OpenSpec services. So we want to be a unified user interface for all the OpenSpec services that are integrated into the release. Obviously with the big discussions that scope may creep out a little bit but for now that's still the range of what we're trying to accomplish and then the other big item is we want to be extensible so we realize that we can't deliver all functionality that you would need to operate a cloud or provide your end users via Horizon. Every cloud I know of runs other extensions and things that they want to manage so we want to be able to have that extension mechanism so that when you go and set up a cloud then you can plug in other functionality in Horizon. So just a little recap on Juno. Some of the big additions were support for Sahara. Sahara came out of or was added to the integrated release in Juno. We have rich support for creating clusters, creating jobs to run those clusters, to do MapReduce and other data processing operations. So that was a big addition for us in Sahara. There's a lot of functionality there and I encourage people to check it out. Better plug-in support. We had a primitive plug-in mechanism that we added in Leap Ice House. In Juno we flushed that out a little bit better allowing you to pull in more of your static files, things like JavaScript, images, other things that allow you to do a more full-featured extension. This also helps operators move away from the model where basically you have to pull down the Horizon repo and then hack the code inside of Horizon to extend it. That's really not a great model for anybody who's trying to deploy code and then keep it up to date with Trunks so we've been working hard to provide a better mechanism. The next item that we did was UX updates. So we kind of walked into some older technology, some older versions of Bootstrap for our styling and JavaScript and other things. We finally got past that so we can pull in a lot of the bug fixes from upstream, a lot of the styling improvements, a lot of the UX improvements. So now we're currently up to date with Bootstrap and a lot of the other underlying libraries that we were dependent upon. So that's been a great step forward. The last big thing I'd like to point out is Plants added the metadata catalog concept and the general release. Horizon provides support for both viewing and updating metadata properties on things like images and volumes and post aggregates. So we provided a nice widget for that and that functionality was added in Juno. There'll be further enhancements of that in SQL. So what's next on the deck for Kilo? So the first item is going to be support for Ironic. So Ironic graduated in the old mechanism again with the big tent model things may change but we plan on supporting Ironic in the Kilo timeframe both for end users and also for operators. So end users will have the option when interacting with nobody to set up a bare metal instance rather than to have that option and then operators as well to both specify new nodes and then bring them in allocation. So with the summit we spent a lot of time talking about how we were going to provide better user experience going forward. That's been a big focus of mine since I became PPL and trying to take Horizon and make it a lot more user-friendly. And so about a release ago we decided to start moving towards AngularJS as more of a client-side architecture. The reasoning behind that is just to provide better feedback to users quicker. Try not to do so much data loading on each request. So push a lot more of the UI onto the client-side cache data there and be able to operate on a lot more data and provide that information back to user a lot quicker. So in Kilo one of the first things we're going to do is work on a reference implementation for how Horizon is going to move forward in this transition. The focus of this will be on the identity dashboards or identity dashboards. So panels like projects and users and when you bring in Keystone V3 you'll get groups and roles and domains. And so this basically will use that as a way to set up reference implementation for going forward on how we're going to use Angular and how others can use it and extensions going forward as well. Part of the improved user experience effort we're working on is improving the table experience. So right now the state of the each server has different APIs and Horizon kind of thinly represents that API to the user via a list or table. So the filtering is inconsistent currently. Pagination is inconsistent. The different APIs do different things with pagination. They provide different filters, different ways to search essentially. So what we're going to do is we're going to move a lot of that onto the client side as well. We're going to try and cache a bit more data on the client and provide the pagination and the filtering in the Horizon code rather than relying on the APIs to do it. In that way we can provide a consistent user experience. So the user doesn't have to guess if they're moving through the Horizon UI and landing on a new page and trying to figure out what the paradigm is for this page. So again, this is just trying to improve the user experience but more information to users, fingertips and allow them to find data a lot more quickly and accomplish the tasks that they're trying to do. Additionally, as far as the user experience, so we implemented a wizard. I believe that was back in Lake Havana. I thought that kind of that stagnated as we were trying to move forward but it's a primitive wizard. So a step workflow essentially. We're going to refocus on that in this release. Part of that is with the Angular implementation. So in this release, we'd like to get the launch instance workflow or wizard greatly improved. That's one of the biggest usability issues with Horizon right now is that the launch instance workflow is just very confusing, requires a lot of knowledge up front. So we've been working with the OpenStack UX scheme for the past release essentially on design to get a better workflow on that and move forward. Part of that, that'll be implemented on the client side as well. And so we want to make sure that we're not kind of painting ourselves into a corner on the implementation. So we're going to do the network wizard as well just to make sure that we have an extensible format, an extensible widget on the client side that we can reuse. We're going to continue to refine the plugin support. As I said before, it's really important that we have a clean plugin mechanism that doesn't involve editing the source code in the Horizon repo in any way or the Horizon package when it gets shipped. Right now, the plugin mechanism does require you to add files into the directory where Horizon gets deployed. Obviously, that's not optimal and it causes a lot of problems when you want to go and update a package if you're doing it with a packaging system. So we're going to continue to refine that so that you can point to where that plugin mechanism is again. And then we're going to have to refine a little bit to provide better Angular support as well as we're moving more in that direction. Better steaming. So another big concern for operators is that they don't want to ship Verizon looking the same as everybody else's UI, especially if you're putting in front of customers or you're wanting to do refiller models. We need to be able to theme it better and we need to be able to do that without hacking into the Horizon code as well. So the goal for this is we started down this road. We started pulling out the appropriate variables for coloring and size that operators want to change to change the look and feel. We're going to continue going down that road and provide hopefully a much better steaming mechanism that is a lot easier to leverage. Federation and single sign-on. So we're working with the Keystone team to provide federated authentication. Obviously, a lot of corporations that use OpenStack have an authentication back in already where all the users are set up. They don't want to have to pull that into Keystone or all that user information into Keystone. They don't want to have to duplicate all that. So we want to provide the federation a mechanism to do that federated log-on. Come along with that. Come single sign-on where if you authenticate, hopefully if you're authenticated into your corporate authorization system already, then when you come to Horizon, you wouldn't have to re-authenticate. Multi-domain identity operation. So Keystone v3 has been out since Savannah. Actually, yeah, I think it's Savannah. There's support for multiple domains in there. We support that to a degree in Horizon. We need to do a much better job. Part of that is you want to have a much richer set of roles than admin and member to do these operations. You want to have, say, a domain admin. We can't really support that right now because we don't interact with Keystone the right way. So we're going to tackle adding that support in this release. So that should provide. This will be a step forward also on the Paracocole multi-tenancy path at the Keystone implementing in CELO. So this work will be leveraged in that as well as what's already there in Keystone v3. Another big thing that we've been pushing on for a couple of cycles is getting integration tests going. So for the long time, Horizon had just unit tests. We had one integration test that ran in Tempest. It still does, actually. But we've been working heavily on trying to get an integration test suite set up and going. So we actually have quite a few integration tests that run against the Horizon gate jobs now. They're not as big as part of Tempest because we're still trying to work through the kinks and get it more full-featured. But I think we'll be ready with that at the end of CELO to be able to run all the gate and jobs, at least for Horizon, I guess. And that is all I had for CELO. We'll continue to finish fleshing out the API support. So Horizon has pretty good API support across OpenStack, but there are certainly parts of the API that we don't represent. And as services move forward with the versions of the APIs and adding features, obviously we like to pick those up as well as we move along. So that will also be... I mean, that's a consistent focus of ours is to try and remain in sync with the other services. And you're free to contact me on free.no.irc as David's type in Lyle, or you can email me at david.lyleandintel.com. We have weekly team meetings. The time is awfully so. It's best to check out the OpenStack meetings page if you'd like to join in and provide some feedback or ask questions. Let's send it back to Al. Thank you very much. Awesome. Thank you, David, and thank you again for your time today in providing your Kilo updates. Next, and last but not least, we have John Dickinson who's going to be sharing his updates on Swift. Hello. Thank you for having me. These things are always fun for me to do. I love the time frames when we're talking about summaries of what's going on because it gets a chance to see exactly what's... reflect on what's happened and how we can go forward from here. So my name is John Dickinson. I am not my name on IRC and Twitter, and I am the project technical lead for OpenStack Swift, which is the object storage component of OpenStack. The background image that I chose here is one of the international prototypes for the Kilogram, and the principal one is stored near Paris. And since the summit was there, that's what we named the next integrated release in OpenStack. We called it the Kilo release after that international standard that is stored near Paris. So I think that was kind of a cool choice, and I really liked that, and I think it actually reflects a little bit of what's going on inside of the community overall and something that I have seen inside of Swift as well, and I'll come back to that in a little bit. But as a high-level summary, Swift is an object storage engine. It is built for scales, optimized for durability, availability, and concurrency across the entire dataset. My vision for Swift is that everyone will use it every day, even if they don't realize it. We see this in a lot of places. Swift is perfect for unstructured data that can grow without bounds, things like content generated and consumed by web and mobile apps, backups, documents, videos, pictures, that kind of stuff. I want to see people using Swift when they're helping their kids with their homework by looking stuff up online. I want to see people using Swift when they're filing expense reports, when they're checking their bank balance, when they're watching videos online, all of that kind of stuff are places that are great for Swift and in fact where Swift is being used today. So my overview is a summary of what's going on in the community. It's not dates or schedules. It's more of a reflection of conversations that are happening. And so that's kind of what I want to share with you. So looking at who's going on in the community, first just kind of an update of the community side of things before we get into the technical piece. Who's participating in the community? I think it's always an interesting question. And so we still have the most prolific and active daily contributors into Swift these days are familiar names in the community. Swiftstack, Red Hat, HP, Rackspace, Intel, all of them are contributing into keeping Swift healthy and pushing it into new directions. And it's a really solid base of contributors, and I'm proud to work with them. All of them are really solid, strong contributors technically and strategically. And it's just a pleasure to work with these companies and to see both what they're doing in the marketplace but also how they're bringing their varied expertise and experience of different use cases into the development community itself. And so that's been really exciting for me to see continue in this recently and then going forward in the community. So what's next? What are we seeing? Next slide, please. I think something that's really kind of cool is that we're seeing a lot more uptick and things that I'm hearing are a lot more of Swift being used and supported by third-party ecosystem apps. I worked with Swift Stack and I have interactions with people in the ecosystem quite a bit. And the thing that I am hearing unprompted from people is that there's basically two object storage APIs that matter today, S3 and Swift. And that is something that's really tremendously exciting. And I was talking about the Kilogram is a standard that people adhere to. And I think what we're seeing also is the OpenStack APIs as a standard beyond what we like to tell ourselves internally in the community, but actually hearing feedback from people who are not involved in the day-to-day minutiae of OpenStack itself but actually just looking at this from a very broad ecosystem perspective. We're seeing OpenStack APIs become supported as first-class citizens. And so one of the things, a few things that I guess are very interesting I've seen recently, is looking at major applications like Commvault and Beam that are used in very large enterprises today. Supporting Swift as a first-class citizen, Storage Made Easy is one that has been adding a lot of great support recently. Glatinet, in fact, I just saw this morning, announced a tighter partnership with Helion Swift, and that's in addition to some of the old favorites that have been in the ecosystem for a while like CyberDoc and Expand Drive. And of course there's a ton of people using Swift internally for internal apps that are writing their apps to talk directly to Swift. And so this is really kind of cool because what we're seeing is that Swift is becoming the thing that people are talking to for object storage. And this is because, and as a result of this both, we're seeing more and larger clusters of Swift in the overall installed base. This drives demand for some of those more traditional enterprise features. And it of course means that people need things like more automation, and I'll touch on some of these kind of things that we're talking about in a little bit. And so in addition to that, in addition to these third-party client apps, but I think part of the same story is that we're also seeing in the community more integration and interest from other storage providers, both with a native integration using Swift's extensibility points to integrate natively, but also their own support of the Swift API. And so all of that is very interesting looking at both who's contributing into the community, but then also the very large ecosystem of people who are building applications and products and businesses around Swift itself and based on Swift as a standard object storage API and implementation that can store the data that people are using for applications today. And so that actually leads me to an interesting new development that we've got in the community right now. How do clients interact with Swift? So we have been talking about how to make things better for applications that are using Swift. And a lot of times the very first interaction that people have with Swift is using the official command line interface, the CLI, or the official Python development tool kit called SDK. And so these things are part of the code base that we maintain as a Swift project inside of OpenStack. And so we've decided to make some changes in some of our direction of what we're going forward here. They're kind of a long-term effort. First off, the CLI is something that we are going to continue to keep and maintain and add features to and fix bugs in and add additional polish to and all of that. It's something that is a very useful tool for a lot of people to be able to interact very easily through the command line with a Swift storage endpoint. The CLI uses the SDK. That is the Python wrapper to the OpenStack Swift API. The API itself is simply using HTTP, so standard HTTP verbs and response codes. But we've got a higher-level wrapper around that written in Python that allows people to interact with it programmatically a little more easily. And the command line interface makes use of those tools. Now there has been an effort inside of the OpenStack community to create a single OpenStack SDK project to give all of the OpenStack projects a more unified SDK and then a single place where people can consume can then talk to all of the OpenStack projects. It's very early, especially for their implementation and their support of Swift right now. But it's rather a fortuitous time for us to very closely work together so that the Swift community and the OpenStack SDK team. And so what we're looking at and what we're talking about right now is taking any efforts for future features and new directions and rewrites and things like that of the Python Swift client SDK and instead of focusing it there we will focus on the new work that's being done inside of OpenStack SDK. What this allows us to do is make things a little more efficient and take the knowledge that we've developed over the last four or five years of writing Python OpenStack for the Python Swift client SDK and taking those lessons and doing it right or doing it at least better this time so that we can focus on things a little bit more like efficiency and performance and stuff like that. So since these guys are essentially starting from scratch and we realize that to do all of these major changes we just essentially have to start from scratch. We figured out that it would be a really good opportunity to work closely together to share the workload but also make something that's pretty cool. So this is a long-term ongoing effort. It is something that is again it's not a particular schedule or timeline or something like that but it's more of a general conversation we're having inside of the community. We of course are going to be very focused on maintaining compatibility and making sure that existing applications that do not break and we're not abandoning Python Swift client right now at all. So we will continue to improve that. We'll continue to make sure that bugs are fixed especially on the CLI side but on the SDK side we're going to look at putting some more effort behind the OpenStack SDK project. So that's a change there that we're working on and some things going forward as far as how clients interact with that. So what else is next? There's another open source heavy group called the GNOME Foundation and they have an outreach program which is kind of an internship program and it's something that is designed to get underrepresented groups involved in open source. So it's a really great goal and something that I'm really happy to say that OpenStack has been a part of in the past and has continued to invest time and effort into. For the current round of the GNOME outreach program OpenStack has six interns participating across the various OpenStack projects and one of those interns is working on Swift and is focusing on improving operational tools and it's something that is really just a I'm really happy that it's happening and our intern for inside of Swift is jumped in and has already made some contributions into Swift and will continue to add in some tools specifically designed to do some operational checking to validate OpenStack deployment or Swift deployment just to make sure that everything is working well. So that being said as far as where the community is and who's working on what and kind of the ecosystem around it and the changes we're making on the Swift client I want to go a little bit into a new process that we have called specs and what that is meaning for how that's been framing the conversations that we've been working on. Specs are basically a collaborative design process for bigger features within Swift. There are several parts of the several different projects inside of OpenStack that have adopted some sort of specs process and it's something that is kind of caught on like wildfire but I think it's very early in figuring out how does this process actually work so it's good to think that this is an experiment. The goal here is to increase communication. The goal is not to have another process to do something so this is kind of our second try at making a specs process for Swift that is effective and works and so our current thing that we're talking about that we're using inside of this integrated release cycle is to when a spec is proposed we want to iterate and land on those specs very quickly. We're not going to look for this big design document upfront that has every possible contingency and use case and corner case defined and explored out. What we want to do is we want to have it as a way to improve communication especially when people are working across different companies, across different time zones and things like that. So we want to be able to we want to land things fast as soon as we can agree on something and we want to keep those updated as we iterate on our conversations and figure out what the best way going forward is and as the coding happens so once we agree on this is a thing that we think we should do then when coding starts we want to keep the spec updated based on our keep the spec updated based on what the implementation actually does. So another point of this is that specs are not docs. Docs are different so they have a different audience. Specs are designed for the communication. So that being said, what are some of the things that we're talking about next slide? What are actually what's actually going on inside of the community? What's being worked on? What are these collaborative conversations that we're actually having? Well, one of the first ones is a big one that we've been talking about quite a bit. Erasure codes. We have a major feature that landed several months ago inside of Swift called storage policies. Storage policies allow you to configure how that is stored inside of your cluster at a very fine, granular level. One of the things that we can build on top of that is to be able to actually configure how your data is stored across a set of hardware. So today we use replication, and then we're building the feature to allow you to do erasure codes on top of inside of your Swift cluster alongside of replication. So the data that needs to be stored that way can be. So what's the status? This is a little screenshot of the Trello board status as of last night. Basically, the progress is good. We are working very hard, and lots of people at lots of different companies are constantly talking about this. We are getting something that's going to be demo-able very, very soon, and a lot of the work that's been done up to this point has been spent on some very hard design problems, and making sure that we have these kind of fundamental problems figured out just before we start doing some of the more demo-able things like, let's just read and write the data up front. So one of the hard problems that we're figuring out that we're planning on being able to support as of this point is the ability to overwrite objects that have been stored as erasure coded data. This doesn't sound like a very hard problem, but when you think about the fact that the different components that make up the erasure coded object are stored across a lot of different pieces of different physical hardware, the coordination problem becomes rather tricky when figuring out, well, what is the actual instance or version of the object that you are actually trying to keep and maintain and know that that's going to be your durable copy. So that's a hard problem that we're trying to figure out, and it's complicated by the fact that SWIFT does not have the concept of a global lock. So you can't lock the cluster down or even lock a particular object down and say, okay, well now we're going to have something that's going to be basically an atomic operation, or even just an operation inside of a transaction that you can roll back. That doesn't actually work inside of SWIFT as a distributed system, so we've got to figure out how do we make progress on this and allow the erasure coded objects to have the same semantics as replicated objects from a client perspective. So we're working really hard on this, and these are some very tricky problems that we're trying to solve and it's going to be done when it's done. This is our top priority inside of our current integrated release cycle, and I'm pretty confident that we're going to have something that's pretty cool pretty soon. Next slide. One of the other big things that we have seen inside of the community, some of the other conversations, especially coming from some of those big customers, and I'm talking about some more of these enterprise features, is the concept of encryption. And encryption is, especially when you're talking about storage, has it's a very loaded word that has a as soon as you say it, a lot of people assume they know exactly what you're talking about. The problem is there's a lot of different use cases that can be wrapped up in this concept of encryption. So what we're talking about right now and what we're hearing from users and looking at the thing that people need to solve first is solving the problem of basically, if a hard drive walks out of my cluster either maliciously or inadvertently, I need to make sure that nobody can read the data that is stored on it. So there's a lot of different ways to do this, and right now we've got some very interested users and customers for this, some active conversations including a very active spec document being worked on right now, as far as here's how we can have a framework by which we can implement encryption which means encryption at rest, the data at rest for data stored inside of Swift, and something that will work for different use cases that will not be too complicated, but also not be too limiting for people who need to do this. So it's something that is, again, an active conversation inside the community. Next up. Next up is this thing called composite tokens. It's a new functionality to work with auth systems. Basically the idea in here is that we have some data inside of Swift that may be put there by a service. So in this sense, especially looking at a broader OpenStack deployment, you could imagine that somebody is using OpenStack Compute and then using OpenStack Image Service to store these images inside of Swift. So you don't necessarily, though, want to have the end user be able to delete those images without going through some other process because that could make some things inconsistent with the whole index and everything like that. So you still need to be able to track it and you need to have appropriate permissions on it. And then on the other hand, you don't necessarily want the system itself to make decisions on the behalf of the user. So the proposal here is to require for certain levels of permission that the operation requires two OS tokens, one from the service and one from the user. So this is called CompositeToken and it's something that has been worked on inside of Keystone and there's very active discussions happening inside of Swift, although there's not any code in Swift for that right now. Next up is something that we call FastPost. Post is an HTTP verb and inside of Swift it's used to modify metadata for objects. And in almost all cases right now it is inside of Swift it's implemented as a copy. So you send a little bit of metadata and the system will read the old data and then write out the new data with your new piece of metadata. So that works pretty well, especially since in most use cases posts aren't done very frequently. Unfortunately it makes it a rather expensive operation. So FastPost is something that is an improvement on that so that we do not have to do that server-side copy. This is something that there's a lot of historical reasons for why we're doing what we're doing but suffice to say it's something that there are some active patches and reviews going in right now to see if we can get rid of that server-side copy so that we can make post operations very, very fast. And that's something that's, again, along the lines of continued efficiency improvements that we always like to look at for Swift. Which brings me actually to kind of the general catch-all next topic here, which is we're working on efficiency improvements inside of Swift. This is something we always want to keep our eye on, especially as people are storing more and more data and having more and more users on this. Remember Swift is designed for massive scale massive concurrency across the entire data set. So we have to make sure that we're the things that we're doing we're not doing too much and we're not doing them too slowly. So there's a few things that kind of go under this right now in addition to the FastPost feature. Some of the stuff that we're talking about that people are playing with and again conversations that are happening right now involve making very, very large containers more efficient. There's some improvements that I'm pretty excited about that are along the lines of improving efficiency in global replication. So if you have regions that are separated by a continent or separated by an ocean, making them a little bit more efficient. And then overall we're of course just continuing to fix bugs, make it easier for end users and operators. This even comes back to the current projects that our GNOME outreach program in turn is working on making sure that we've got some operator tools that really allow you just to validate your cluster very simply. So it's something we always want to keep our mind on and again there's lots of conversations with that. So would you like to get involved? That is if you would like to get involved you're more than welcome. We would love to have you help out and come on in. You can get involved. The best way is hang out in the OpenStack-Swift IRC channel on the FreeNote IRC network. There's a lot of people there in a lot of different time zones. So generally any time of the day or night you should be able to find answers to your questions and figure out how to get involved. We even have a wiki page that has just some lightweight ideas of something that we don't even really have any detail around it but somebody has mentioned it more than once and it would just be kind of a cool thing to work on. So it's even kind of a good place if you'd like to play around with some things and you don't have your own idea of what to work on already this is a good place to start with. And then if you'd like to begin coding and get involved with actually making the code itself better, fixing bugs and features, things like that the best way to get installed with a dev environment is using the link provided here to Swift all-in-one environment that can be very simply provisioned with VirtualBox and Vagrant. So we would love to have you come along. We have weekly meetings, those you can find in IRC. Also you feel free to contact me Twitter, email, IRC and I try to respond to as much of that as I can. And that's what we have. So if you have any questions, please let me know. And Allison, thank you for having me today. Awesome. Thanks, John. And those links that he has in his presentation will also be below in the YouTube description. So again, if you'd like to get involved please visit those links and contact John if you have any questions. And that is it for today's overview. Again, please reach out to any of our PTOs online today, David, Owen if you have any questions or you'd like to learn more about how to get involved. Thank you.