 Good morning everyone, just give it a few minutes until maybe more people join. Okay, it looks like we have seven people on the call. So anybody has anything that they want to bring up or they want to talk about besides what's on the agenda? Okay, so if nobody has, nobody else has anything, so the main item on the agenda is the QH incubation presentation. So Kevin and Yin, are you guys on the call? Yes, I'm on the call. Just give me one minute to see if Kevin wants to draw. Good, thanks. Thanks for running the show in my absence, Ricardo. I must appreciate it. No worries. As you can imagine, I've had one or two things going on the last little while. Yeah. Work is keeping you busy. Yeah, Facebook has a couple of things going on at the moment you might have guessed. Yeah. Hello. Hi. Kevin, long time no see. Hello. Hi, Peter. And then it looks good. Yeah, I don't know why my personal account is not able to log in, so I'm using the QV edge account. Do you need me to share? Yeah. Yeah, I cannot access Google Doc. Yeah, let me share our doc. Yeah, one more thing. If you guys are attending, so please sign yourself to the ad as an attendee. Yeah, can someone please help me to Yeah, I'm going to read the document. Yeah. Okay, so I think we can start. No, the present button is on the right top. On the right. Yeah, the left of the chair. Yeah, this one. Yeah. Okay. Okay. Thank you, everybody for joining the meeting. I'm Kevin Wan, and Ying is helping to share the slide. So I will just briefly introduce QV edge, the updates since we've been accepted as since the sandbox project. All right, next. All right. So QV edge is actually built on top of Kubernetes, trying to extend its functionality to the age. So provide the functionality to manage applications, as well as resources as well and IoT devices at age. So it introduced the cloud and age communication functionality to make it work over the limited stable network between cloud and age, and also provide age autonomy functionality to make the age able to work when it's disconnected to the cloud. And also QV edge did some optimization for low resource use cases, especially for IoT. And also QV edge provided some extensible framework to able to connect IoT devices to make it to simplify the communication between application and devices. Next. So this is a high level architecture of QV edge today. So basically, QV edge want to provide the functionality to make a Kubernetes cluster, managing the nodes in the cloud as well as nodes on the age. So in the middle, you can see there are mainly two parts of QV edge. So in the cloud, it's the cloud core. Actually, you can consider it's kind of a operator to do shadow management for the applications and nodes and IoT devices on the age. The next page, I will give some more details about the things inside the cloud core. So on the age, we have the age core. So basically, it has a lightweight Qubelet inside, and we introduced the metadata persistency for node level to make it able to work when it's disconnected. So currently, it's able to integrate with the standard container runtime and as well as container network and CSI. And especially for IoT cases, we introduced a standard extensible framework. So on the graph, you can see it's a mapper here. So basically, it's a protocol adapter to convert between IoT protocols and MQTT. So for cases that an application need to communicate with multiple IoT devices, it can just rely on MQTT instead of being aware of Modbus or the other the protocols. So currently, QV edge provide implementation of Modbus, Bluetooth and OPC UA. Ideally, we want to make it able and easy for any user to integrate with their own IoT device protocols. So this is the details today inside the cloud core. So basically, we have three controllers. The age controller is actually doing the shadow management for the Kubernetes core APIs like node, pod, config map and secret. And device controller here is actually a IoT device controller. So QV edge for any user introduced a set of CRD to represent the IoT device. So to help simplify when the user developing applications to communicate with the device. So the device controller here is to deal with the life cycle of that device CRD and also do the shadow management to synchronize all the relevant information to the edge. And the single controller is recently we added to, so basically it's to do a periodic check to reconcile just in any case the data between cloud and the edge are not up to date. So the admission web hook is currently doing some CRD validation. And in the long term, we want to also add some best practice enforcement for age scenarios. Kevin, can I interrupt and ask a question? Or would you like me to keep it till the end? I think either is fine. Yeah, Clinton, just go ahead. I mean, but I thought. Yeah, the reason it's useful just because the diagrams up now. So I'm kind of curious how you guys deal with inconsistencies. So in particular down at the edge, you have a bunch of software running and potentially you can modify the state of the nodes down there. But then you also have controllers up in the cloud. And it's possible, you know, not only for them to get out of sync. So it's not just like behind, but but actually that there are mutations happening on both sides. And how do you like reconcile the conflicts there? And what is the mental model that people should have? Should they only control things through the cloud? In which case, it's not controllable when the edge is disconnected, or should they control them at both sides? In which case, how do you reconcile the conflicts? So actually, like all the metadata, especially the spec, are just come from the cloud. So the agent order will just follow every like the spec part of the Kubernetes API object. And the agent order would only update the status. And currently, a little bit different way made in QB is that we actually didn't reuse Lista watch between cloud and the edge, because that that's one that one is designed for the data center network. So you can find out that we actually introduced the cloud hub. And also in the agent order, we have a edge hub to deal with the connection. So between that, it's a web socket. Web socket is able to do by a directional communication. So the model in QB edge is actually every update, every information update is is sent by by the cloud. So so for the edge, when it persistent any updates successfully, it will reply a act. So we will check the if the act successfully received. And we think that's that one one time synchronization is successful. Otherwise, we will retry. And also the sync controller here is just the recording every object, the version number, when every time up to update success. So it will compare to the the original object inside the API server. If there's any version number difference, it will think that it's kind of inconsistency and and send additional update information. Thank you. That answers my question very well. Perfect. Thank you. Welcome. So for the QB edge CSI driver here is so the for the CSI a little bit different is that the upstream CSI model integration model is also done designed by in the data center network. So here we think for CSI in the age cases, the whole CSI backend would be on the age. So so the CQ age CSI driver here is trying to hook some some requests and the response to the to make it send it to the back end to the age. Then with this plug in QB edge is able to integrate with any existing third party open source CSI drivers from upstream. So you have storage devices storage volumes both in the cloud and at the edge and they're controlled by different one is the QB edge CSI driver and the other one is the normal cloud drivers. Yeah. Yeah. Yeah. So in the cloud the the user can just install the way they did before. In the age cases they need to install a whole system of CSI in the in the age and and then install the QB edge CSI driver and it will help to send all the uh request to the CSI backend at the age. That makes sense. Thank you. Can I ask a question? Yeah go ahead go ahead Dan. So if there's like a firmware update that happens on one of these edge devices how is that all understood from the cloud side? Like you know one of these uh and it's one of these tiny devices or even larger devices that they firmware update because of some security problem or something. So yeah so currently for the IoT devices management we only deal with the communication part so that means uh when this when a device is ready connected to the to a node the user can add a custom resource to to tell the device details to the system. Then the application can reuse that information to to connect to the device. We not yet covered the like a firmware update in the current implementation. Can I add something? So actually we are in this class with Eclipse hotbed project. So basically they handle the firmware update because when you update the firmware you need a full backup of a whole OS on device in case the firmware update failed they need to roll back. That's provided by the device OS provider. So we so our plan is to collaborate with a hotbed to do that the do the firmware update with the on the device. That's a question so um what is the use case for having storage in the cloud and also the use case for storage and at the edge? So actually there are different applications so so in the cloud the the user may just want to have some to the applications like today they have in the data center and also in the cloud like so like if the multiple there are multiple application instances inside a age site so they want to share some data and also the using CSI on the age they can persistent some data like for IoT like industrial IoT they have a lot of metrics information from the sensor so they need to they want to persistent persistent that data in the age so that one if they send it to the cloud it will use out the network bandwidth. So would video be one of the the applications against for storage at the edge so people may be trying to get more video content faster at the edge yes yeah and then how would that communication happen between the cloud and the edge is I mean you there's this same communication that looks like that's happening through WebSocket but did you mention also that Quik is also supported between the yeah so so currently WebSocket is a underlying implementation to between the cloud and the edge and also we implemented Quik as an alternative but according to the test result there's no big improvement and WebSocket sometimes is more stable so currently WebSocket is the default protocol okay thanks sorry to and Ricardo maybe you can keep an eye on the time and just shuffle us along I don't know how much else we have on the agenda besides this but please do feel free to shut me up if we start running out of time well I'm just wondering yeah we don't sorry we don't have anything else on the agenda so we have the whole middle for this oh awesome so mobile phones are in a in a way kind of special cases of edge nodes they have most of the properties of an edge node and unreliable communication with the cloud and they also have a bunch of sensors and cameras and various other things have you guys found any use cases or given any thought to using this for mobile phone kind of applications especially given what I was interested in mobile phones we are doing a a inference offloading project a collaborative with a cranial community so in the future slides Kevin will talk about our goal so basically the co-pad will focus on two directions one is iot the other is MEc use the case so that MEc use cases are mainly for the technical provider or that provider the edge a mobile edge in the technical cases so connect mobile phone to the data center okay so MEc is mobile edge cloud no it's used to call a mobile edge cloud now it's called a multi-access edge cloud because they did that mobile phone you can switch between the wi-fi and the you can switch the wi-fi or other methods you can switch back and forth so you can now say oh it's only on the technical network but you could jump around so but there's still need to be but it's basically mobile phones connected to the cloud okay also the self-driving cars especially in the new new generation of the technical network coming so one more application will fall into this okay now let's move to the next all right so this is what we have today in the edge node so basically we have a edge hub that's to deal to talk with cloud hub to to to manage the the messaging between cloud and edge and also the meta data manager is the to deal with the node level meta data persist persistency basically it will persistent part config math and also the the node information and the secrets on the edge so the hd currently we are actually removed the sum of the packages of from the vanilla uh cool it so currently it's a it's basically a lightweight uh cool it in the early days we we we did some inline uh code change but why we are in the middle way to uh to reduce their inline uh code change anyway that's a detailed thing so for the device screen that that one is to to to actually so the the concept is from iot scenario so that one is to deal with the iot devices when when any uh information like the sensor data come from the iot device it will to duplicate the information uh one persistent uh on the edge node and then another one send it to the cloud so that one also uh represents on the uh device crd so the difference here is currently we only actually deal with or we recommend the device train to deal with like the uh switch and the sum uh like uh kind of static uh properties from the uh uh device because that that one the uh the data uh don't have a much data so it's it's uh it's easy to uh synchronize for the heavy load if there's a the the iot device is uh producing a large amount of data we would uh recommend and user to deploy a tsdb to also and also integrate with the other iot pipeline uh middleware to to kind of reduce the data and pick the useful things and the edge mesh here is actually uh uh the uh service and the and the node level dns implementation implementation here so we try to uh simplify the uh service discovery and the service communication uh on the edge especially in cases the node are located in different subnets the uh the communication between uh the subnets would be a uh big challenge the event bus here is actually to deal with uh basically the events so uh converging from the uh raw mqtt the raw mqtt structure to the device train yeah okay next so uh for the community growth uh growth uh i want to highlight that currently cuba is uh uh becoming uh more and more healthy so uh here are some numbers comparing to the uh early days when we uh being accepted as a send the bus project you can find out the uh number of contributors are now uh we have more than 300 so actually uh in for those uh the total contribution we have more than 100 so get up start so we have now uh uh 2500 and also uh uh forks uh we uh grow six times and the maintenance currently we have the 14 maintenance so maintenance here including the uh maintenance and the approvers and the contributing member organization uh that's a big improvement uh the early days we have only one and now we have more than uh 25 and also uh we are currently working on some of the uh cross community collaboration uh like kubernetes iot age working group and also we are uh closely working with acrena we have uh there are two blueprint there uh using uh cuba age yeah and also the under discussion is the uh eclipse foundation yeah on the right uh these uh there are four uh top new contributors uh in during 2019 and uh you can see our uh contribution chart is steady going up so very steady there's no uh big gap so very steady development yeah okay so uh this is a bit about the uh contribution and the adoption so you can find out there are different types of organizations and the companies contributing to the cuba age including the iot and hardware companies like arm samson and also uh we have telecos so uh currently they are all from china and we are also under uh uh interaction with the other uh telecos and also we have it service providers uh and also cloud providers contributing to the project and especially we have uh some academics joining the project as well so uh so for the uh user adoption currently we have uh more than 20 but only a few uh got confirmed to make public so i also listed them here okay next so uh this is one so this one is actually currently the biggest uh the largest uh end user adoption uh the end user name we are still under confirmation but i'd like to share the uh some of the uh adoption information so basically this is a uh is a uh highway uh etc uh ecosystem in china so the user want to build the the whole system uh whole system based on the cloud native technologies so the architecture is there what is what is an etc system uh electronic toll collection it's just a high uh freeway toll it's more like on your uh bridge yeah so sorry it's like a fast track or something on on the highway when when somebody's driving and automatically charges you right yeah no i understand that uh i just didn't know what etc was thank you all right uh so the uh so the whole architecture is that they uh they want to manage the whole system by a central cloud so they can uh currently they are they are using uh cuba cubanettis and the cuba to manage the uh the toll application and the distributed to the uh the etc gates etc stations in the different uh province in china the all the toll data would go into their own toll system in in their uh unprim unprim data center so the biggest challenge here is the uh the actually the the etc gate is really far away from the central cloud data center the network is uh very unstable and the bandwidth is uh very limited so uh qvh helped uh to make it much easier because it's able to uh to synchronize the information bit over uh internet and make it able to make especially for the age nodes it's able to work when it's disconnected to the cloud so uh currently uh they are using uh qvh to manage uh more than 50 000 age nodes and also there are uh 500 000 containers in total currently so the currently the the inside the containers are mainly the toll applications uh they have plan to uh to do uh to develop the vehicle infrastructure uh cooperative uh cooperative system and also that one would use the uh cloud native technology as well so for the age nodes the underlying architecture uh they have the uh most of the uh age nodes are the uh arm can servers and also they have some uh x86 ipc so ipc is uh industrial or pc it's kind of a small pc so for the uh business uh perspective they are currently managing dealing with uh 300 million data records every day so uh for the for the uh the benefits the from a driver perspective the the time used to passing through the toll system uh toll station you can find out it's uh it's a big improvement for the uh the car it now only take two seconds because they don't need to stop and manually deal with the uh the building stuff for a truck you can find it's it's uh it's an even larger improvement okay next so uh this one is a uh a smart campus and the user so uh basically they want to uh to make the uh the campus smart with some ai functionality so uh the end user uh actually they have they already have have some ip camera in in the campus and they don't have uh the uh the wires for uh dealing with the uh and also the uh much power uh redundant power stuff for the uh making it smart so the q-wage used here is to the way uh they use q-h here is they have the uh they have some room to uh to manage some uh pc and the can servers so they install a q-wage there to manage some uh containerized applications and for the camera because uh it has the uh interface to uh to to expose the video stream through an ip address the applications uh can can uh can get these uh get that video through the ip address so in the cloud because they are not uh the the underlying server is not that powerful on the edge so currently like the uh the uh high level analysis is still uh doing in the cloud so basically on the edge they are the q-wage is helping to manage the applications to do uh like face recognition and also flow analysis like how many people are entering uh a gate uh per hour like that so for the uh uh end-to-end benefits uh this uh this solution helped the campus saved around 30 percent uh replace the low work because they uh they need less security guards they can just rely on the uh camera and the ai applications and also for like the visitors entering the gate uh it's much smarter uh with the ai functionality so it's uh it just needed seconds to for a visitor uh entering the campus okay next but a quick comment here i would imagine that this is a fairly big uh use case where for ai related systems a lot of the inference happens at the edge and and you have um constantly updated retrained models that have to be pushed out to all these edge locations uh regularly and sometimes constantly so is that a is that a use case that you've seen for these kind of deployments yes yes yes we are seeing that and also that's why we uh uh collaborate with acreno there's a called uh machine learning offloading uh framework developer uh based on kube edge so that's the uh we talked about in the previous slides is acreno kube edge offloading service in the that blueprint is running on the acreno community based on kube edge very very cool and we can set up some demos if we need so we can show how we use a mobile phone and offload the emotion recognition to the edge then if that's not enough we uh we have the constant train model push from the cloud to the edge so that's the demo is uh just early in the early stage but the final release will be in the early q4 but we already have the first edition of our demo ready so we are going to set up a environment have a edge behind firewall we have the uh a cloud set up on aws to demo this environment so then you can use cell phone to access the edge server and yeah something like that the the cloud running on the public cloud your server running behind the enterprise firewall but your cell phone can connect to the edge to offload your uh inference requirement the inference thing yeah i think it would be i think be really useful to to provide that demo both at uh the the upcoming kube cons there's the one in china but i guess the us one is towards the end of the year again um but also perhaps do a very condensed demo maybe five minutes or 10 minutes to the toc um i think there was a fair amount of interest in in using kubernetes in very very different ways than it was originally designed for because this is not what you know the kubernetes designs had in mind but it's uh clearly very useful for these kinds of things um so i think we should we should really um kind of get the message out they're not all the details necessarily i mean this is a great presentation for this group um but i think just you know one slide with the architecture and then a quick demo of of a very uh sort of easy to understand use case would be super valuable in getting the message across thank you thank you for the suggestion if uh we have time at the end of meeting i can show a quick uh overview of that project the architecture yeah nice thank you okay let's move on all right so this is uh okay so this is a brief history about the project so uh before uh 1.0 we actually moved to 1.0 in 2019 so before 1.0 we are actually focusing on the uh fundamental applications like so mainly it's to provide like with age components and also to provide core api support on the age so and then we we are trying to add some more useful functionalities like uh so also like we we verify the container d integration so for cube age itself the uh the component in the age currently takes uh 70 megabytes and also like recently in 1.3 we verified the cryo integration so cryo takes only 30 megabytes so that makes it possible for cube age to manage uh very small can servers like uh probably uh even down to 256 megabytes servers okay so i won't go to the uh details about the uh features so yeah let's move to the next yeah i add one thing it's uh from this year we're already into the regular release cycle so we have a three month release cycle it's kind of a one month behind the upstream Kubernetes release we're going to incorporate the new release of Kubernetes integrate then we release cube age so it's about a month after a formal Kubernetes release so we are also in the quarterly three month release cycle now so very regularly that will help all the community member understand our release cycle and have a uh their contributions uh they are aware of when they could be alive and integrated into the main branch yep okay uh sorry i interrupted there i hope i didn't miss something i had one quick question um so how do you and this may have been covered i just got interrupted by dog um so so you clearly there's there's a lot more to cube age than than traditional Kubernetes and there's a lot of custom components and things but there's some amount of it which shares like apis and stuff with with parts of Kubernetes um how do you deal with keeping the two in sync is this is this a just a hard fork of Kubernetes at some point and you never plan to to sort of synchronize the two or or do you have some plans to somehow keep the the Kubernetes parts in sync with with more recent versions of Kubernetes so uh actually uh in early days we are hosting uh some of the inline changes uh for the kubelet because we want to make it smaller on the edge so uh during last uh few months we have moved it the uh more uh easier way so we are we are currently vandering the uh the kubelet uh code and so basically we rewrite the code entrance to to only uh load the packages we think that's necessary on the edge and for for the api support that's uh easy we just rely on the client go yeah so currently it's don't have we don't have any chance to client go so we keep sync with upstream kubernetes api we don't make any change only a few changes on the kubelet side sorry maybe i wasn't clear so i'm i'm talking about the cloud side as well as the edge side so there's a whole bunch of stuff in the cloud the api server not the client i don't think yeah go ahead no uh so on the cloud we it's a additional set of controllers so that that's not uh no we don't need any change to the uh the kubernetes control plan uh components so the only thing is that we currently reconstructed the uh kubelet to ignore some modules when running it on the edge yeah basically out there is vanilla kubernetes that's that's great news thank you yeah well we don't have we we don't change the upstream cloud side of kubernetes we just uh we we just uh take the upstream directly however we don't uh blindly we just i said to take a month because we want to see if there's any uh new apis if we support or not so basically we only extend the cloud api we don't change any of them so we are compatible with upstream all the time we don't have any hard fork on the cloud side excellent thank you uh question so do you have a support matrix uh so one of the problems with kubernetes i don't know if it's a problem but is yeah we do they okay so they release every three months uh now or so four releases a year uh and now they're going down to i think three but uh you know one of the challenges is you know keeping track with that api and the changes in the api right so so i guess you have a you have a matrix already that identifies this and says uh this version is it's uh works with uh one we're not as 115 and 116 or whatnot okay yeah i just shared that's our uh compatible matrix so that so we keep track of that's why we uh kind of a month behind the upstream release so we fully compatible with the cloud api from the kubernetes so we just we don't fork we just use it we just wander around here okay thanks okay sorry can you go back to the slide yeah yeah i'm looking for that steeper sorry give me one second there is go yeah the window is gone just give me one second i can project if you need to yeah you can i can i'll find a window here sure i'll get it it's only one page left oh yeah i see something's broken in zoom recently seems to do it easily yeah because uh today's June 4th the Tiananmen Square anniversary there's a censorship in internet in china okay so here i see it doesn't allow me to uh there you go yeah i can open it if you pan out yeah yeah i'm struggling as well i don't have the link to the slides yeah it's in the meeting notes i i can share now okay we can see okay um yeah okay so uh for the uh follow-up plan uh from the community perspective basically we want to wider user adoption and also um yes next yeah and also uh we are uh looking on to attract more uh developer to joining the uh project development and also uh we we did some we did some advantage evangelist uh last year and this year uh we are also joining like the uh the google summer code and the community bridge to uh help more uh promote the uh project and for the uh so actually for uh for the community governments we uh this year especially we want to enforce uh the six so uh currently there are the two are about to get started one is the uh the uh device iot or iot device but we already have some uh uh participant uh companies joining the discussion uh discussing about to improve the uh device crd as well as the device mapper framework and uh provide a more uh use for uh reference architecture to simplify for for end users uh when they try to integrate their own device protocols and for me see uh so because this it's kind of uh uh different between uh with the iot so for the mec six currently are mainly the technicals joining to discuss uh how to uh use q wage to provide a uh the uh common uh underlying infrastructure to enforce the the whole mec system so yeah so the kind of status of these two seags are are discussing the uh the single charts and probably will be announced in the following month and uh for the technical thing uh we especially uh so for the two seags we we definitely will uh do some uh more development and add some more advanced features uh to uh better serve the uh device iot and then me see but in general we want to integrate more uh to do more integration with uh cnsf projects as as well as lfh projects and also we uh we will keep working on uh to improve the reliability and the security to better uh serve the end user adoption yeah we have the uh project uh roadmap link and uh so in the last i think uh we need the uh help from cgron time to uh recommend for uh incubation in the uh cnsf that will be uh help the project better moving forward okay uh yeah i think there's more about the uh matura sorry didn't mean to interrupt that's all that's all about the matura yeah i haven't i haven't spoken to the rest of the sick about this but but just based on purely on what i've seen here i know as you know quite a lot about the project from from years gone by when i was working at hallway um i mean this looks like a perfect candidate for incubation to me it's uh it's clearly got some momentum it's clearly not you know uh mature enough for for finalization for graduation yet but it's it's a great candidate for incubation it has enough momentum it has a bunch of companies involved uh and and it's in use in in you know production in at least a few use cases so it sounds like a very very good incubation project to me and it's also the first that i'm aware of anyway the first major uh kubernetes project which is sort of a very specific specialization of kubernetes that that was not really intended for which i think is is interesting in itself yep i fully agree with that i think it's uh thank you yep i think um thank you next steps will be to um start working on the document recommendation due diligence document yeah do we have any volunteers does anyone have the bandwidth and the knowledge available to be able to do a good job of having a look at this so to be clear we need you to go and look at the code maybe you know install it uh try it out um talk to a couple of the pretend customers if possible and and just make sure that uh i mean our job is to make sure that what kevin's told us is all true that's one of our jobs uh if it is then i think this is a great project but somebody has to just go and dig around a little bit and make sure that uh that this is all um as it seems so uh do we have any volunteers for that any volunteers stun silence all right i can i can volunteer to do that then um let me see kevin do you have a sort of an envisaged timeline i imagine well when is the date for kubecon china i don't remember it's july so it's july 30 to august the first okay so so that would be one obvious deadline where we would like this thing uh to be in incubation by the nis you yes it's about a 45 to 50 days yeah something like six weeks um so we would need to give that you know i don't they're they're busy working on what the process is and i haven't been following that discussion too closely but i would imagine that the toc need at least a couple of weeks just to kind of open this up for public scrutiny etc so unfortunately yeah it looks like it looks like we'll have to have this due diligence done ideally in the next two weeks i'd say in the worst case by the end of june um just to give about two weeks for a public review and then the toc can hopefully vote after that so um rikata maybe you can um just get in contact with diane's left already hey is that where she's still here okay um so so what we need to do is just make sure that everybody's kind of ready and knows what the timeline is otherwise this thing's going to get delayed and we're not going to meet that deadline so i can i can do my best to try and crank out a brief sort of uh due diligence document in the next let's say three weeks um but we we do then need to make sure that we have the dates lined up for public uh review and toc vote before uh kubecon china starts so rikata maybe i can ask you to to line all of that stuff up and i'll focus on the due diligence unless there is anybody else who would like to help with that yeah sounds sounds good yeah yeah um well maybe we can think of offline and coordinate and how we're gonna go about it so sounds good yeah yeah absolutely i'm gonna run uh thanks for a great presentation kevin and yin uh that was very interesting and congratulations on making such great progress in the last uh what two years i guess thank you yeah thank you thank you for the uh the advice yeah all right thank you guys thank you it's all efficient