 Thanks so much for being here, really appreciate the time and our nice little drive around Detroit car interviews So why don't you tell us a little bit about why you're here at? Kubecon while I avoid going to Canada Because it's super easy to fall in the trap of getting in the tunnel apparently you can't get out again unless you go to customs I've heard this and I'm loving the signs that say tunnel to Canada. Yeah, it sounds more like an instruction Right, we want we want more immigration come to Canada Yeah, exactly. I still like the thing I tweeted about yesterday. It's like I still can't get over the Canada's south of us Which is like wait like every because you know, I'm a east coaster, right? So the water is always on the you know, east side of me And having it on the south side and it's going to the wrong country. It's very confusing my brain Yeah, somebody did say that so they refer to the Canadians as our neighbors to the south. Yeah, right, right, right Yeah, I think being an American. I think it's a little bit more weirder for me, right, but So yeah, so tell me about Kubecon. What are you doing here? Yeah, so here all week So I've been doing ebpf day yesterday. Okay, that's it Then today we had a city and project meeting this afternoon Got a talk at service mesh con and then you know the main show right right week So and so how was the ebpf day was there like we're doing like a workshop or like was it more like meetings of the team or No, ebpf day was a co-located event. So we had presentations from people mostly end users I would say actually projects as well. Yeah I mean wrapped up with a really great panel of People it was run by Frederick His name begins with L It might be large in law but from TechCrunch and He had this fantastic panel of people talking about You know using ebpf in production people like New York Times and Google and Bell Canada nice speaking Canada. Yeah. Yes. Yeah, right and did you get like the turnout you were hoping for like was it You know all the people it wasn't bad. Yeah, I would say we actually had a bigger turnout in Valencia But I've got a feeling that there's more co-located events. Yeah, there's a competition. Yeah. Yeah, totally Yeah, there were a lot. Yes. Yeah, I was just looking at the list and like So many choices so many choices So the project meeting you're at this morning, what is what are you, you know focused on or what are you kicking around for? Yeah, so this was a new thing for for psyllium. I don't know if other projects have done it before but basically since he I've offered us a meeting room and and some time and said go for it So we publicized beforehand a document and said sign up here if you've got like a PR You want to discuss or a problem or you know, maybe a talk you want to give and you want to buy anything really and To be honest not that many people signed up and then loads of people turned up Yeah, we had a pretty packed room. We had to keep going out and getting more chairs, right? And yeah, some good discussions are going on. I've snuck out to come and do this Yeah, but we definitely appreciate it. It's we thought it'd be a lot of fun and so I was gonna ask so yeah, all I can picture is like a deli counter, right? You've taken the numbers for all the people who want to talk about things It's always hard to like I think although sometimes when you get a group of people who are Interested in a discussion around things like PRs or problems are having or whatever The conversations that just kind of spontaneously occur are almost better than most other scenarios because you have a bunch of people are really engaged You know, so so hopefully it'll Yeah, there were definitely some some good little you know groups gathering and talking about because we had a whole range of people who were new to So yeah, I'm who wanted an introduction and then we had people who had very specific problems that they wanted to discuss or PR that they wanted to get some help on so yes, it's some good conversations. Yeah. Yeah That's cool. Have you been to Detroit before? No, this is my first yeah, and what do you think have you enjoyed? Actually really nice. Yeah. Yeah, I have a friend who lives out in the suburbs took me out on a beautiful bike ride on Saturday, so I got to see all the beautiful trees and some amazing Like the architecture. There's so much for I yeah, I was I was really surprised by that myself. It's funny I am I'm staying in the What if they call the Renaissance center here that they're out. I'm told it's called the Rensen. Yeah I see that all over the place But I I walked in the building I was like this looks really familiar and I remembered I actually did a sales call to General Motors Like 15 years ago to the on-star and and it was in that building and I was like, oh, that's why it looks familiar But yeah, it is so different now Like kind of the city in general and I was talking to some of the locals and they were kind of saying the same thing It's just like how how much it it seems to have kind of come back But there's still a lot of you know a lot of open space, which you know, like still still waiting for recovery from the pandemic Maybe yeah, yeah, I definitely seen it a Variety of you know, you can you can see there's a lot of Disparate levels of ink, right? Yeah. Yeah, totally But yeah, it's pretty cool. And you know, I do like, you know Detroit's kind of not really in it anymore, but you know, you get the fall colors really well Up in kind of the northern. Yeah, you know, I think it's beautiful. Yeah Yeah, and but you said you actually took your bike ride out here on the Isle, right? Yeah Yeah, we drove about an hour outside of Detroit picked up a bike that I could borrow and then rode all the way back in Navigated the hotel and then came out to Bell Isle and rode around around here Well, at least you got to get a sense of the, you know, where the venue was Which is always nice I'm I have a panel on Wednesday and I was like trying to find on the map where exactly the talk is So I think I've got it, you know, I had to like color it in on the map So I could like remember but are you are you giving a talk this time? Yes, well, I'm part of the Cillian project updates to okay So and and that's a pretty straightforward set of updates and then this afternoon at service mesh con Oh, yeah, I'm hoping to have a demo for that one. So that's always a little bit more jeopardy. Yeah. Yeah Yeah, I have you have you taken having like recordings as a backup. I I've never I think I've done it once But I'm always like I'm just gonna go with it. I always so I'm going to have a recording, right, right I have about six hours to go. So there's still Yeah, yeah, I'm sure it'll be fine It's always more exciting to do it live anyway And you always what's nice though is that the crowd's always with you because they're just as terrified as you are And yeah, so everybody wants to see right and they want it to go well, right, right? So, you know, you at least you have a soft crowd normally So what it what will you be demoing? So service mesh essentially Cillian service mesh And we have a nice integration with some Grafana metrics. Okay It's a very simple, you know, but a demo app that's sitting behind Kubernetes ingress set up to be Of Cillian type so right Cillian service mesh is providing that ingress functionality and under the covers it programs envoy and then Get a nice service map inside of Grafana dashboard and get some really good metrics. Nice. Yeah I one of the things I it's funny on the on the insider show that you've done if you know You did before but also in the old level up show I used to it's like one of the things that Is so hard about using these, you know, kind of monitor distributed systems, right? It's like you really need those visuals to be able to wrap your head around what's actually going on Because it's so because the whole idea of it, right? It's it's distributed and somewhat disconnected But that means your brain has to like connect it all in your head and that's really hard So yeah, I really appreciate things like Grafana You know, which really at least for me help me wrap my head around it again But uh, yeah, so it's cool. You were saying that there's a bunch of new integration with Grafana. Yeah Yeah, that's sort of new news this week that Cillian and Grafana are Working together and we've got some really nice Integrations so not only can you export Cillian metrics into the Grafana dashboard, but we also have some Grafana Graphs and service maps and so on visible from inside the Hubble you are. That's cool So it works both ways right if you want it in the single pane of glass Yeah in Hubble it's there if you want it in the single pane of glass and Grafana is that right right? So whichever one you're like already committed to yeah So one of the things like especially things like service mesh Can you give the kind of audience a brief run like what is a service match like why do I want one? Yeah, so At one level you think but surely Kubernetes is running applications and they're networked together right like They're already talking to each other. Yeah, the service mesh is giving you this kind of application layer layer seven Additional features so things like retries Load balancing so you can load balance across multiple instances of Whatever back-end service it is MTLS is a common feature that people want to see from service mesh and Observability is a huge thing and right you know But I think one of the things that we're quite keen on with Cillian service mesh is that a Lot of this functionality can be provided by the network and from an application Developers point of view they they shouldn't really have to care what's happening at right different layers of the network It should be my my service wants to talk to another service and I need this Kind of configuration or this level of retries or I want to roll out this canary deployment and So we're kind of seeing it as Three layers now. There's the data plane itself actual network connectivity passing packets around there's the At the top the configuration, you know, do you want to configure an ingress? Gateway do you want to configure gateway API? There's a lot of work going on in the It's called the gamma initiative gateway API sort of new version that will support service mesh abstractions like HDP routes, right? So you configure that and then in the middle there's What you could call a control plane, but increasingly I don't think that you know application developers need to care about right But this is the thing that's kind of wiring up how Proxies are configured. Yeah Kind of Kubernetes in general right the service mesh kind of in more specific, you know one of the things I really appreciate, you know being a long-time, you know application developer and I guess we're not going to go that way because the road is closed and here is a goose Yeah, who's busily crossing the road But we One of the things I really like is this kind of idea of we just want I just want to give You know the system a hint about what I want right rather than having it You know the number of times you have to go through an install guide and say okay So this is where the database is and this is where the web server is and here are the ports They're going to talk across and all this stuff and I don't like I just I just wanted to you know I just want to kind of describe it and then let it figure it out And so it's one of the things I really appreciate kind of about them You know more this kind of modern idea of like you're just going to describe the system You know, I really think it gets back to like even like puppet really was big part of this where you had this I'm going to describe what I want, you know, and then the software actually makes it true, right? You know which at least for me is is much better and then on top of that keeps it true Which I think is a part people don't fully appreciate how hard that is over time. Yes Yes, that kind of state reconciliation thing, which I think is the the thing that makes Kubernetes What it is right a level above the things that came before it is this constant reconciliation of Here's my desired state. Here's the actual state, right, right and not relying on humans to actually make that state Yeah, right. Yes, you know, which like said, I I really appreciate it You know, the other thing is I think it also Let's us concentrate on you know things that are you know for lack of better term higher value, right? So we can start to think about hey Can I make this more like an event-driven system, you know Or you know in integrating like serverless and all those things because I can I can just put my code where I need to and then You know, I just have to kind of configure it and then it just works, right? Which I think is huge With the service mesh. Have you been? You know, I guess one of the things I'm still trying to wrap my head around is like I feel like Multiple Kubernetes clusters is weird Like as in I I want it to be just one thing But this is a I guess kind of an ongoing struggle is like there's there's people on both sides of this fence where it's like No, I want to have all these different clusters, right? And I but I want to treat them as one or you know, whatever Do you have any kind of feeling about this might just crazy that I think that there should just be one So it's one of the features that a lot of users really like about psyllium And it's almost too easy. So it's called cluster mesh, right? And you have your services and you can annotate them just say well, this service is a global service Mm-hmm, and you have to go around and say for each cluster like, you know, here is My service, I don't know. I'll call it tree, right? I'm seeing a lot of trees. So here's my tree service You know, and I want that to be a global service and in another cluster I've also got the tree service and I will mark that as a global service and Cluster mesh kind of communicates these shared services I keep running in the closed roads Like wait, so first of all the navigation should be helping me with this problem second Why are all the roads closed? This is the adventure of doing the interview in the car is that we'll interrupt for navigation problems Oh, yeah, I'll get back to cluster mesh. Yeah, so you have these services They're essentially the same service. They just happen to exist in more than one cluster And you can mark them as you know, I want kind of local affinity. So if there is an available Instance of it running locally, so I'd prefer to use that But if there isn't I will use one in another cluster, right? And then it as I say it because you're just annotating services. It kind of seems a little bit too And I think that's a really great example of how the network can provide a lot of functionality For you that users shouldn't have to worry about you know users shouldn't have to sort of think about Like which bits of network are talking to each other, right making it at a service level, which is pretty much an application level is That makes sense to me. Yeah. Yeah, no that I definitely agree. I mean it all kind of comes back to that I guess the Like that's a super nice feature because I think that's at least for me right part of my problem about getting my head Wrapped around it. It's like I don't I don't want to think about the fact that there's multiple clusters, right? I just want I want my service available all the time As close to the user as possible and you probably want those multiple clusters so you can do things like yeah As close to the user as possible or like availability zones or right right, but yeah, yeah other than that You're not deliberately saying this this one has to go there Yeah, I mean I suppose you might want to have different services in different clusters for isolation reasons, but then you probably You know the multi cluster thing is less of a You know, they actually don't want it. Yeah, you don't really want them communicating in general Yeah, and maybe the you know the air gap model, right? Yeah, but yeah I think I think the most common example. I've kind of heard it's around security but you know at least for me like I you know I worked on a system which had You know it was one of those ones where it was used around the world But its biggest load was at 9 a.m. Ish right at whatever time zone because that's when everybody logged into it And so to make it actually perform we actually had all this complicated stuff and this was well before communities existed But all this complicated stuff to spin up a bunch of hardware, you know essentially at 850 Whatever time zone so that you know they'd be able to handle the load and then you know and then spin it back down again and Being able to you know essentially You know you have not only this kind of management of where is the service available or the functionality available? but then also like You know can you build in scheduling such that that's also as sophisticated, you know, can you you know? I really want to see one of the things I've been Toying around with this, you know essentially getting into like AI machine learning for scheduling You know which I think would be super interesting because I want to you know I want I want the computer to notice that there's all these logins at 9 a.m. And preemptively spin up a bunch of containers to prepare for that because I think you know it seems obvious to me One of the first things I did with containers we had a startup that was called micro scaling and it was this idea of auto scaling before yeah, yeah, and using kind of control theory, I think it was called, you know Kind of trying to predict, you know based on how demand's changing like how many more right right gonna have to spin I like how fast is demand changing and based on that? Well that I mean that's kind of the problem with current auto scaling, right? Is that by the time the auto scaling starts? It's too late Yeah, you know, it's like no you have to you have to guess for him when it's gonna happen And you know guessing things like what humans are gonna do is such an easy problem, right? You know, of course completely joking And you might be better off with the rule that just says, you know at 850 will spin up right right double the number of instances So coming back to KubeCon a little bit Is there any particular talk or you know is there somebody you're really you know? Glad to have seen or going to see that, you know is a you know a particular driver for this particular KubeCon for you I'm not gonna call out anybody specifically, but it's just so great to see people in person You know I've already run into a few people that I haven't seen in person for a couple of years And it's great to have the opportunity to see people face-to-face again Well, I was some of that in Valencia, but there's a whole whole lot more. Yeah. Yeah Yeah, well, I ran into Chris Short yesterday Who I did a twitch show for like a year and a half every week with him I was on that we were on the same team yesterday was the first day I met him in person. Wow. Yeah, so it was super It was super interesting We of course had had the discussion about the fact that he was super tall with the last name short So I was at least expecting him to be quite tall But you know because I'm not that tall I'm like 5'10 right and so so that was interesting But yeah, also, I think I've now I think I've now met more people here that I've been talking to for the last two years That I had never met in person then I have met people in person that I have met in person before Yeah, yeah, it's super complicated, but yeah, it's not crazy though. It's so weird Yeah, and not everybody's here because yeah, we're still in three shows The main event kicks off tomorrow. Yeah. Yeah, I mean, it's you know It's still easy to kind of go and register for your daily sticker But yeah, it's it's been shaping up pretty cool I wish I was staying the whole week, but I have to get back to give lecture on Thursday So yeah, so I can't stay the whole time, which is too bad But you know you do what you can it's funny because now, you know I've moved out of industry into academia about whatever a year and a half ago and The schedule is all wrong You know, it's like it's really funny It's like the academic conferences are kind of lined up with the school year And all the industry conferences are like a right smack dab in the middle of the semester And so it's kind of interesting and it's kind of what I've been trying to figure out is like, you know Is there is there a way we could you know fix this somehow because I would like to see more kind of academic people at cube con Yeah, you know, but like I know for me I had to like get coverage for a lecture, you know, it's kind of a big deal for me to travel right now Yeah, and so you know, it's kind of interesting in the ebpf world. We have It seems to me anyway that there's a lot more overlap between what's happening in academia. Yeah. Yeah, and we quite often get submissions from Students or graduate students and you know people who are working on I don't know PhDs doing things with ebpf. So yeah, it's definitely You know an overlapping interest right, right, no, definitely I mean, you know, it's like one of the things that I never really knew anything about till I joined boss university Is this concept of multi-party computation or secure multi-party computation? Which is Like I was like, I've never even Like talked about this before but so there's a couple of experts on my, you know in my department um, and what it is is like um, it's I want to compute something like, you know, the average Of a set of numbers, but you don't want to share your information with me And I don't want to share your my information with you and I don't also want to share any information with a third party Like there's no holder of the information. Right. Um, and so Uh, it's basically so what you can do is you can kind of the example that was given to me And I don't tell it very well, but um is like Say we both got a grade on a test, right? We want to know how the class Will how we did as a group on average So what we can do is you take an your number and you add an arbitrary number to it And then kind of tell me the total and then we kind of go around the group And then you keep adding it up and then you can take the average of that and you'll get the right average Once you withdraw all the arbitrary numbers Right, but I never see your grade right and and we don't have to have a third party that we trust that holds the grade either Right, um, which is, you know, what I think of as the norm normal way of doing that, right? But yeah, so they did a It was a like a front page boston globe article using like some of the professors from pu to do the Basically, it was a bunch of research around the average pay for women in boston versus men Okay And obviously, you know a lot of companies didn't want to share that information And so that was the technique they used there was another one too that I think is super interesting is they Boston was trying to figure out how to make ride-sharing safer in that when they pull over To like give them spots so they wouldn't be basically double parked in the middle of the screen. Okay. Um, and uh Uber and Lyft, they wanted their data, but uber and lyft don't want to share their data with anybody about where their drop-offs are But so they could figure it out by this same technique by using, uh, you know, I don't know exactly the details of how it worked, but um Yeah, and so I found out that um the bar that's like five blocks from my house Is the single biggest drop-off location for uber and lyft in the entire city. Wow I was like, I knew the place was popular, but like holy, you know No wonder there's all of these cars all there, you know and lying down the street I've not come across how you do those calculations, but I have come across the problem a little bit in So I'm involved with this organization called open uk, which is trying to advocate for more use of open source and open data and one of the I guess meetings that I went to there was a discussion about Yeah, how You might have essentially sovereign data or industry data that you Want to be able to collaborate on without actually sharing the data. So yeah, that's exactly what this does It's super neat. The I mean there are some challenges about like what kind of results you can get to But that's where the research is going. Like, you know, they're trying to solve as many of those as they possibly can You know at the moment But uh, yeah, so you can do some calculations, but you can't do everything you might want to but yes still I was like That is it's like brilliant. It's really interesting But uh, yeah, so if open uk wants to get involved with So part of the other hat I wear at the u is we do this experiential learning program called spark Where students work on projects for like external parties. So like it ends up being mostly like local government and nonprofits But like one of the ones we're working on is for this organization called museums moving forward, which is trying to First estimate the equity at museums right now just on like staffing But I think ultimately they're interested in like art and stuff as well And so they're going to use this mpc technology to do so And our spark students, you know, so undergraduates and graduate students are building the software So it's kind of cool But speaking of open uk as well. I just saw a tick tock from a guy running for mayor of toronto Who was advocating? Much more use of open source And that's like part of his platform. I was like, that's very cool But yeah, that is cool. Was he thinking of it as code Data transparency more as um, actually of of like saving money Um of the you know, like why are we why is every city building the same software? Right, you know, uh, which I am a strong proponent of as well Um, you know, it's actually it was funny. I worked on a system years ago for the state of new york Um, and I don't know if it's still true, but if you build software for the state of new york It's required by state law that any other place in new york can have the software for free Because it basically it's open source within the state And uh, which I was like, why isn't that true for like any software that gets built custom, you know for a You know government entity, right? You know, because taxpayers paid for it, you know, um, but uh, I don't know if I'm supposed to like wait for this light Or because there's no there's no other way to go, right? Yeah, I'm going yeah, I thought at first this was a line of cars waiting, but no, they're just parked there See it's always an adventure when we drive When you've done this a few a few more times will be a little simpler. I uh, yeah I think my my experience with driving around boston makes it uh, so that I have a little bit more experience with craziness Um, all right. Well, uh, why don't we wrap the interview and just kind of say hit thanks so much for coming and we really appreciate it. Uh, and uh Yeah, absolute pleasure. Thank you for tour of Bella. Yeah