 Okay, we're back here with Cosmin Lehenne from Adobe, our next guest here at the exclusive coverage of HBase Conference of San Francisco World Live. This is SiliconANGLE.tv's exclusive coverage of the Hadoop ecosystem. I'm John Furrier, the founder of SiliconANGLE.com and thanks for watching. So HBase Conference, tell us what you think about the event and so far and what do you think about some of the talks? Once I was impressed with the number of attendees here at the conference, I think the first Duke conference I've been to, there were like, I don't know, three or four hundred people while this is the first HBase Conference and we already have six hundred and a lot on the waiting list and I've seen some really interesting presentations such as the Facebook presentation this morning was really inspiring and along the day I've been seeing very different use cases for HBase. We're also doing a lot of work in this area but being able to see all these people and what they're working on is really inspiring. So you're at Adobe, so we're going to get into that in a minute about what you're working on at Adobe but I wanted to ask you, what are you seeing here that's jumping out at you on those use cases? I mean, I would agree that Facebook was very inspiring and really, really cool that they're sharing. I know they just went public and usually hardcore about not sharing stuff so really thought that was really good form by CArctic. What other use cases are you seeing that you're learning about that's impressive? So I haven't actually considered HBase for example for use cases such as search indexing like full text search and it seems to be a use case that people are trying to invest in. Another one was some real-time backends for mobile applications that seem to be really interesting. We're mainly doing real-time data storage and analytics but there seem to be other interesting use cases. Yeah it's interesting as the community starts sharing their practice and you can learn a little bit more there and I'm seeing names of companies here, Fidelity Investments, Adobe, eBay, Google, Facebook, Twitter, Netflix, Amazon, all the main tech players are here and it's a lot of production discussion. I mean I'm finding that to be something, do you feel the same way? Like getting stuff in production? Yeah we've been running HBase and Hadoop in production for the last four years now so I think we're some of the veterans of the technology but seeing companies such as eBay moving on HBase is really interesting. I mean we have use cases generally where small to medium use cases with HBase but I've seen some really large deployments out there and yeah Salesforce as well has has an interesting use case with and a nice deployment. Of course it's even though we've been using it for four years now seeing what everyone else is doing is interesting. Just talk about what you're doing in those so four years your veterans in the space I think you know batch you know that produce could work back then take us through the evolution of your environment because now real time is a big part of the conversation here and this is not possible a while back just even a short year ago so you know a lot of maybe some custom deployments but going full production like Facebook is impressive take us through your your evolution from four years ago. Yeah so four years ago I was asking people from other teams to land a server so I could build the first Hadoop cluster so that was that. You were begging for some machines. Yeah I was I was actually begging for machines because I needed to I had some some big data use cases and I wanted to try it and then we we've started the providing internal services based on Hadoop and age base mostly age base so we initially we realized we can't scale our MySQL deployment to fit so many so much data. We had a use case with 40 million user profiles so we decided we're gonna use age base and starting from there we started using it more and more so we then we used to index image data such as images on some of our online offerings and then we started doing analytics on the data so we did some machine learning and then we figure out we need to we need some fine grained access besides Hadoop for machine learning so age base was was again a good fit and then we just provided this type of service internally for other for other products so right now we have both data storage system and an analytics system on top of Hadoop and age base and the real-time story of course for some of the use cases we've had that from the beginning like the real-time data stores but for for analytics that's somehow new to us as well so what we what we did initially we started running smaller map reduce jobs and then just move toward just doing it in real-time so what's been the big evolution for you I mean is there a point in time when you say okay this has really changed the game a little bit go back what year did you say okay age base is going to be really much more of a production environment for you was it four years ago when did age base really become a key product was it four years ago so for four years ago we started using it in production in the meantime age base yeah age base we I think we I mean I might have been some of the early implementation yeah I think we still have a cluster which we've never upgraded after that because it's been running then then we were phasing it out but it's still it's still running version 18 of Hadoop so it's like Asian now in these times the computer history museum yeah and at one point I mean we were building these internal services and at one point we just switched and did only this storage and processing of the data so I'd say probably 2009 was was was a flexion point so to speak and when we we did everything with with the biggest challenges that you've overcome with the complexity of obviously some programming involved in some this attack he asked these open source but still early it's still early in the community even today these big holes you can drive a Mack truck through these holes and you got to kind of fill them up so there's nobody else so so you got to write your own code and kind of didn't get it back when with our some of our first deployments were actually backed by by my sequel so we had both date the data both in age base and in my sequel so we would have been able to switch from one to the other so data safety was was one of the first thing that we we wanted to nail down we we could have downtime but we could never lose that was a mirror not a replication was it replicated or just a mirror it was a lazy mirroring yeah it's we would take backups and then we would restore them in the other system so but that was initially because we didn't trust the system enough so we could we could only rely on that but and after that we started doing our own testing so some of our tests would we would just go in the data center and unplug machines take disks out of machines and so on and then get back to to the console and maybe fix the box so we've done that in the past it's a lot of work it's a lot of work brute force I'm happy when we don't need to do that anymore okay so what are the challenges you see today so going forward the world's mobile it's cloud big data is analytics it's becoming much more mainstream in the business side as well as in the application developers so what's your what you see as the challenges and the opportunities so I mean street is picking about the age-based challenges our biggest problem right now is that you gotta be a what what we call it DevOps to operate it so one of the challenges is to be able to actually give an entire environment to the operation people and let them handle it and so we could actually focus on developing stuff on top of it so in terms of use keys is moving towards providing backends for mobile mobile application I think is a challenge because it changes the way people interact you don't have you don't have a website we have a lot of people maybe texting or or or doing some other types of interactions among themselves what is your take on the whole discussion around MongoDB Cassandra and there's different approaches you got Neo4j J4 or whatever it's called so there's different approaches obviously this is politics involved in some of the open source communities but in general age-based is actually related to Hadoop so it's kind of buckled bundled together but also early enough where it fits some use cases but it's fresh is that an issue was it more of just the community you know what's all the hubbub about this versus Mongo versus Cassandra every every moment then we we end up having this type of conversations when we started using age-based we we tested the Cassandra and hyper tables as well that was before the no sequel term was even kind so we had to we had to test this all the system there weren't so many back then and we chose age-based and worked for us I think it was it was a very good choice why I wanted to work for you guys basically because and probably mostly because of the community that you have right now around age-based because it could have been a good system without the community and it would have been a lost investment right because you don't want to that's why you're using open source but because you want to rely on the entire community so in terms of communities definitely it was a winner and with the community comes the performance improvements and all the features and everything so when when I compare age-based with with other system that's the first thing I'm the first thing I'm looking at now in terms of features of course there are other cool and good systems out there but I don't think there's any other system that can store so much data and be so reliable with so much data than age basis so final question for you what do you see happening for age-based in the next year year or two a year to with age-based so I mean there's a there's the stuff that everybody's asking for like secondary indexes and so on but what I would hope to see for age-based in a year or two is to have easier deployments and basically have it have it running on its own basically being being smarter about the way it handles everything behind the scenes so I won't have to keep tuning it and so on so it would be a self-healing system it shouldn't be too smart in this area because you're still in some control but there's a lot of stuff that you shouldn't do so I'm checking some questions we have on Twitter here and getting some DMs from people saying hey you know John good questions ask some tougher questions about application developers not necessarily you're selling to them but you're one of them right so you're a computer scientist my friend wants to know I love age-based non paraphrasing 140 characters love age-based good stuff I'm so busy it's just too hard to use what's out there for me I want to really start jamming on coding I'm an application developer I love what age-based can offer me what's out there for help me long we the questions basically saying hey I love age-based I want to use it I want to do a Facebook did I want to play with it I want to develop on it I want to take advantage of someone else's success what's out there right now to make a little easier to code strictly speaking with age base there isn't much out there age base is a product which is currently being used and meant to be used by I don't let's say advanced users but what I know and what I've heard and I of course I can't talk about any details I know there will be platforms that will offer these type of services you know that to be true yeah you've heard some things I hope to come on some things yeah so but even so I mean I'm not sure why anyone would say I need to use age base you don't need to use age base if you need age base just use age base otherwise you can just use a service and when I'm thinking about services there's there's Amazon has a lot of services and then if you if you if you really need to to I don't hold your own destiny then you can you can use age base but before you know you need so much data you I don't see well content is building a managed service I know Jonathan Gray's working on some things and some other yeah Jonathan Gray yeah he's he's doing something in this area and I this is the kind of stuff that I'm expecting to to see growing and then probably be able to use in a year what would you suggest as for someone out there building a platform to advance the development of and increase the inbound migration of more community members what kinds of tools are needed what are the kinds of features and minimum requirements to kind of get that next level of developer to come in with in the context of big data or age base age base taking advantage of the benefits of age base so I mean let me let me make sure I'm getting this right in terms of tools cloud there are builds already really good distribution of Hadoop with age base in it so you can just you can just take cloud there are distribution and error manager works great saves a lot of time yeah so so and they also offer commercial support so maybe maybe these aren't exactly tools but you have a good distribution and you have the support so you can start you can start using that if you want to if you want to use age base besides that I mean there's a there's a good community but strictly speaking about tools I don't I still think that we're lacking a lot in this area yeah I think it's true and then we've been talking about with other guests as well final question how does Adobe feel about this movement if age base is the mood inside the company feeling positive you were begging for servers four years ago well where are you now and what's the state of the Adobe philosophy I mean we're sponsoring this event so I guess that speaks for itself we're moving towards services more and more every day with with our new creative cloud offering and everything so we have acquired a lot of companies that are in the services so in general I think there's a really good feeling about about Hadoop age base is somehow not that well-known internally it's our team and probably I don't three or four other different teams that are using it but we have some we have some really important products on age base and just to give you an example this year we've deployed five clusters so we have nine clusters out of which five were deployed this year so there's definitely a lot of growth in this area so and we're looking to to to build more on top of that so how Adobe is looking at everything big data related I think it's looking looking very interesting in this area okay Cosman thank you so much for coming on the Cube Adobe doing some work congratulations for sponsoring this great event first ever age-based conference really a groundbreaking moment again the community's phenomenal congratulations on your four years making servers to now to really being a leader congratulations we write back with our next guest here on side the cube on Silicon angle dot com Silicon angle dot TV's exclusive coverage of each base conference we'll be right back for having me here