 Hi, John. This is Krishraguram. I work at Intel and I'm representing the OpenStack Foundations Product Working Group here for this interview. So thank you for joining us today. Really appreciate your time. Thank you for having me. And we of course, interviewing you as the PTL for SWIFT. So tell us a little bit about yourself, John. Sure. I've been working on SWIFT for, wow, nearly seven years now. And I have been a part of OpenStack since the very beginning. The project technically for SWIFT and I currently work at SWIFT stack in San Francisco. Terrific. And tell us a little bit about SWIFT itself, which is the object store in OpenStack. Right. Well, in summary, SWIFT is an object storage system that's designed to store a huge amount of data that gives you extremely good durability and extremely good availability, even if you're storing things across a wide geographic area. And so the basic idea is that the whole thing, the whole reason we're here is for applications, right? So applications need to store and access a lot of data. And when you've got data that needs to be accessed on the internet, when you've got a lot of static content, things like videos and images and documents and backups and scientific research sets and those sort of things, end user created data from mobile phones, from playing video games to solving the mysteries of the universe. SWIFT is what you need. A cloud isn't complete until it's got object storage as part of it. And SWIFT is here to meet that goal and to provide durable, available and scalable object storage for applications. Great. Now we've just come off a very successful design summit in Austin. What are some of the hot topics during the SWIFT sessions? The Austin design summit was very good and we covered a lot of ground. And what I really, I mean, one of the highlights for me was at the very beginning and during the keynotes when we've got a deployer in Europe, OVH, gets off on the keynote stage and says, oh yeah, by the way, we've got 75 petabytes of data inside of OpenStack SWIFT and that's on more than 23,000 servers. And that's just really exciting to see that sort of thing and to hear how the community is embracing and using SWIFT in production. So when we went from the, just the basic keynote side of things and walking around and talking to people and seeing what's going on there, we'll get into the actual design summit sessions. And we had a few really great things there. One thing that I like about the summits is that we've got specific time set aside for operator feedback. And this is the people, it's designed for people who actually run SWIFT in production and it's designed for me to get up at the front of the room and be quiet and listen. And to have a lot of the SWIFT developer contributors sit there and listen to what the operators and those who are actually using SWIFT in production are seeing where their pain points are, what's going on there. And so those are fantastic. We had two solid hours of that in two separate sessions, which was specifically dedicated for that. I would say that probably all of the sessions have a good aspect of that. But some of the things that we're talking about there include, you know, how to, the day-to-day operational things. It's like, okay, so how do we make things better when you run out of space in your cluster and you need to expand and how do you get out from underneath your customers trying to put more and more data in faster than you can plug in hard drives. Or how are you dealing with what the logging messages are and how monitoring what's going on and kind of the pain points that you may have seen in certain conditions and the deployment choices that you have made. So that kind of stuff really gets the whole community to hear and understand the kind of the real world practical aspects of the project. And so those are incredibly valuable and I love those things. It's definitely a highlight for the entirety of the design summits. So when we talked about more of the sessions, we went into a few separate, you know, I think we had a lot of other stuff as well. We talked about just general basic, we've got these existing features, how are we going to continue to improve them based in parts on some of the operator feedback based in part on what we actually hear in users feedback from things like well, how are we going to make small objects perform better? How are we going to make sure that Swift Client has the appropriate feature set? How are we going to make sure that the Erasure Code functionality is meeting the needs that people have for it? We talked with some other projects about integration there. One of the integration points that we're working on that I'm pretty excited about has to do with the Searchlight project. I've said several times that if you've got a storage system, which Swift is a storage system, then the most important thing you can do with the data is store it. That's the first thing somebody wants. Great, you have to store it so that they can get it back later. But the second most important thing is that you need to know what data you've stored. So it doesn't really matter if I've got 75 petabytes of data stored if I can't find it, if I can't figure out what's there. And one of the things that we're working with on the Searchlight team is a metadata search integration. So that's what the Searchlight project is set up to do is to integrate with things like the Elastic Search stack and figure out how to put that into there efficiently so that we can do efficient searches from a system that's designed to do that specifically that. And this is something that in fact in the ecosystem that we have seen a lot of people do already. We've seen IBM has something like this at SoftLayer already being used in production. We know that HP has been working on this. We know that SwiftStack has been working on this and using this with various customers. And we've heard other requests from other people as well saying, hey, we need like to do this. How do we integrate this in? In users keep asking us for it. And so what's really great is we've got that pressure from the end users plus the interesting thing going on inside of OpenStack itself with the Searchlight team, which means that together we can come up with a very good solution that's going to help everyone out. So I'm excited about seeing progress on that and seeing what comes from that. I expect good things to happen there. One of the other big things that we talked about that's one of the major things. It's the top priority in the community right now and something that we're expecting to deliver within the Newton timeframe is our encryption at rest functionality. So when you're going to store data, if you need to ensure that that data is stored encrypted, the best way, no question, is that you encrypt your data and then you send it to the storage system. And that's great. There is a need, however, to send data to the storage system and have the storage system itself encrypt the data. And this has some operator benefits that are really interesting. It's not something that's particularly protecting against anybody seeing your data because you're sending the plain text data to the storage system itself, of course. But what it can do is ensures that when the data is durably stored, then it is not able to be read by anybody who doesn't have the proper credentials. So one of the greatest things that this is going to enable is, well, you don't have to worry about hard drives being lost or shuffled or RMA'd or recycled into other servers and exposing data. And this really helps, especially a lot of larger enterprise groups who say that we have to make sure that this is encrypted data stored, even if it's encrypted on the storage system itself instead of on the client, because that allows us to meet the legal requirements we have to store personally identifiable information, medical stuff, insurance stuff, financial things, all of that kind of data. And so this is something that is really going to help increase adoption and remove barriers for adoption or SWIFT inside of a lot of these more traditional enterprise places. And in the community itself, once we're building this foundational feature, we'll be able to add other things onto it. And of course, we're working very closely with the Barbican project to deal with key management and how we're doing the secrets. And that's going to allow some greater functionality in the future that is pretty exciting. And so I would say the last thing that came about in the Austin summits, I mean, there were a lot more, but the last big thing I'll mention here today is that we have had some very interesting, almost research and developments style of work happening for about a year or so now, involving rewriting some of our, some of our back end durability pieces in a different programming language. SWIFT is written entirely in Python right now as are most of OpenStat projects. But one thing that we've been looking at is some efficiency improvements that we get by using a language called GO. And the results have been incredibly promising. And as a community, we came together in Austin and were like, okay, so we've seen this. We've seen what's happened. We've heard it being talked about at the past two different summits. We've seen presentations on it. We've seen people who have in fact even started running this in a way inside of production. And it looks very successful. And so one of the things that we've been doing since the Austin summit is, okay, what is this? And we're not rewriting the whole thing. We're just taking the specific things specifically. We're looking at focusing on replication speed and being able to make sure that the data, that the consistency engine inside of SWIFT is as efficient as possible. We'll be working on that. That will take us beyond the Newton cycle to get that implemented and merged most likely. But it's still a major effort that's now underway kind of in the mainstream of development inside of the SWIFT community. Yeah, you covered a lot of ground then. There is really good summary. Everything that was covered. Now you did end up also identifying specific user needs or problems because you talked of scalability and monitoring pain points, performance issues. And then you talked about the whole elastic search and the search capabilities being put in. And of course you rounded it off with these new features that are not being worked. Would you say then that one of the top priorities for Newton is really the encrypted storage thing? That's really one of the top priorities. That is the top priority right now that we're currently working on and expecting to have it wrapped up within the Newton timeframe, yes. And can you think of any others that would also be high priorities in the Newton cycle itself? Well, that's the... One of the great things I love about the SWIFT community is that the people who are actively contributing and the companies that are actively contributing are significantly invested with production clusters of SWIFT. And what that means is that every day we're hearing from the end users. We're hearing the requirements that people have and we're working to improve the experience for operators and end users to solve storage problems. So if we identify a problem, we're going to start working on that problem and we're not going to wait to schedule it for allocating it within a certain timeframe or it's got to be done by this thing. What we're saying is look, we've identified a problem and we're going to solve it. And when is it going to be solved? When it's done. So the feedback that we have, especially at the summits in person, but also every day throughout the overall community discussion from operators, from production deployers, from end users, and many times those end users being actual paying customers of our respective employers of the broader community. What this means is that, yeah, there's a lot of other stuff we're going to be working on inside of the Newton timeframe there. That doesn't mean that we've necessarily said, well, we have to get this done in Newton or we're going to wait until after Newton to get these other things done. What I've laid out is that we know that encryption has been being worked on for a year or more already and we expect it to wrap up within this before the Newton release happens. We also are going to be working on some of these bigger things like the Golang stuff within the Newton cycle, and we've also got some interesting things going on as well. So some other features, two features including people who are working on automatic tiering of data inside of changing storage policies based on different rules, people who are talking about how to more effectively manage global clusters, how to improve performance with specifically looking at replication and erasure code, rebuilding, optimizing for small files. Those are the things that are going on that will continue to be going on during the Newton cycle and also beyond that. Yeah. And if I looked at all the things you covered and if I outplayed a little bit, it seemed to me that some of your key themes are scalability, performance, and manageability. You talked a lot about monitoring. Is there anything else that comes to your mind as one of the key themes? When I define SWIFT, it always starts with, about storing a lot of data. So from the very, very beginning, SWIFT is about scalability. And so, yes, we've had massive amounts of data being stored in SWIFT today all over the world. But that doesn't mean we're done, per se, with solving the scale problem. We're always looking at making sure that we can continue to move forward. So we have demonstrated that SWIFT is perfectly fine with storing 100 petabytes. But what happens if we need to store 10 exabytes of data in the system? Those are the next level things that we'll have to be talking about as we look to the years ahead and how data is just exploding and there's this huge need to have scalable, durable, and available storage. So at the same time, we also recognize that if somebody has a storage system, it's only useful if it's actually used. So it's got to be in production, it's got to have applications that are being written for it. So those sort of things come down to a lot of the manageability of it. So what is the day-to-day operator life? What happens when you have hard drives fail and servers fail? And how do you need to be alerted and how do you need to keep track of what's going on inside of the system? So there is a high degree of manageability. sort of things. Looking at searchlight integration and Keystone integration and other parts of the OpenStack ecosystem. I think one of the things that's really great about OpenStack in general is that you've got a suite of projects designed to implement a cloud infrastructure inside of your own data center. And the whole is better than the sum of the parts. It's really great to have a good storage system and it's really great to have a really good system. But together it means that you can have what we term cloud native applications that are designed to elastically scale and that's really fantastic. But you also have a storage system that can handle all the data that those scalable applications are generating. And so together and improving that interoperability story and even across clouds that I know is a big part of OpenStack is something that is really important. And so I think these things together, the scalability manageability and interoperability are things that aren't particular things that we're going to focus on, but they really need to be baked into the DNA of the Swift community itself and the things that we're always thinking about with every line of code that we write and every patch that we review and add into the project. Terrific, terrific. So is there anything else you'd like to add, John, to the OpenStack community as a kind of closing pitch? Well, I Swift is amazing and it is amazing because of the community behind it. And I am incredibly proud to have been working with and continue to work with the people who are actively contributing into Swift. We have been growing very rapidly over the last six years of OpenStack. We've recently passed 550 total relative unique individuals who have contributed into Swift with either writing code or reviewing code. And we'd love to have everyone come join in and be a part of that. There's things that we can do that will satisfy whatever needs you have. If you're looking for I just need to solve some bugs, I want to help write some documentation. I want to solve some small things to get involved with an open source project. We have something for you, but if you really want to dig down into the deepest layers of the kernel and file systems and want to understand how those interact across massively scalable distributed systems, we've got those problems too. And we need your help. And so I'd love to have people come help out. My email address is me at knot.mn and you can find me on IRC and OpenStack Swift channel. My IRC Nick is not my name. And so please feel free to reach out to me or ask on the mailing list or ask on an IRC and I will find a place for you within the community. Thank you very much, John. Really appreciate the time and the summary of all the points you've covered on this interview. So thank you very much. Thank you, Krishna.