 Thank you everyone for coming here today we are going to give a talk about the staff motion and here is the agenda for today. First we're gonna explain why we hear what is staff motion and how it works then we have a demo and at the end we will show our roadmap and our future work and some other related work. So who are we? Interesting question. So my name is Tin and we are software engineers and we have been cloud for three contributors for about two years. Since we graduated from the Pivotal Dojo two years ago we have a small team both in Cambridge and San Francisco about like called the DMC Dojo. So what is the Dojo? So the Dojo in Japanese literally means the place of the way where students come and practice martial art. So does that mean we practice karate all day? Is that true? Not quite. We do it sometime but not recently anymore. Actually we have two main focuses. The first one is we contribute to Cloud Foundry projects and also we innovate on those projects like come up with new idea and doing research. The second purpose is we do digital transformation where we adventureized pair programming and test-driven development among the internal teams at the DMC. So in short we have two focus Cloud Foundry and digital transformation right? Yes. So Zubin can you remind me of why are we here today? Why we here? Come on. We're here to provide a solution that a lot of Cloud Foundry users may have the problem that Cloud Foundry may have in now which is like migrating their applications from one Cloud Foundry instance to another with all the services. So what is... Let me give you a scenario here. So I assume most of you guys are developers. If you are not a developer you're probably in the wrong room. Just kidding. So we always have some applications running somewhere. Let's say I have applications running on PWS and my applications highly relies on distributed database and you know I'm really curious about Bitcoin and I actually look into the Bitcoin and figure out there actually implementation which is blockchain is way more cooler than Bitcoin itself because there's another version of how you do the distributed database. So I was thinking maybe I can use blockchain in my own application but sadly PWS doesn't support that yet but happily another Cloud Foundry instance do support blockchain and service. So what should I do? It looks very simple. I just have to push my application again onto the new Cloud Foundry. Yeah you're right totally true but that's only the first step. So normally in order to make this migration you also need to migrate all those services that bind it to your application. You can definitely do all those steps manually but it's kind of like tedious you might have mistakes here some of them might be critical you might have data lose or there's certain amount of downtime. You know Zubin I still don't get what you say because I'm a very chill developer so I I don't mind spending my time repushing the app and redo like five services no problem at all. Yeah fair enough I believe you but let me ask you some other questions. Do you want to run your applications in a very old slow computer? No I don't. Okay yeah. Do you want to have a smaller number on your PWS monthly statement? Yes. Okay fair enough. So for reasons like better performance lower cost the stronger security or compliance sometimes we do want to migrate applications from one cloud foundry instance to another even as individual developers like team care so much about these facts as a company those facts attached to their business a lot might hurt their business. So let's say I'm an operator that in charge of hundreds applications all the applications running in the public cloud because the private clouds we just built has so many benefits as we talked like a strong better performance lower cost the stronger security or compliance we now we want to move all the applications to the private ones so what should I do? Like previous slides we need to repush all those applications again and also all those services bind into those applications instead of doing just like once now I have to do it again and again and again so mistakes could be way more easily made here and any of them could hurt our business may make trouble to our business. So let me count so five times 100 applications it's 500 services no I don't want to do that. Yeah see that's that's why we here to provide our solution which is staff motion so team what is staff motion? So I I got a definition here it's a tool for app mobility among cloud factories. That sounds still very abstract can you give us more detail about how actually it works? Yeah so I prepared a very beautiful architecture slide here hope you guys don't fall asleep. So I let's say that I'm an operator right and I have my cloud factory instant on the left side and now I want to move an application to the right side which is people the web service so I check my environment I see there's an application binding to my SQL service now looking further I saw traffic coming to a load balancer to the app so how do I move this using safe motion first of all I will push what we what we call the safe motion agent on both of the clouds then using the command line tool we say okay move my application then the tool will contact the agent on the left side and backing up the data into a storage cloud storage somewhere and then on the right side the agent gonna pull the data and restore it into a new my SQL instance at the same time it gonna bring up another app like why I'm drinking tea like then it will after everything is done it's by the application to the app and to the my SQL instance and then redirect the load balancer hmm what happened I think my some guys must be thought like me it's like it's very like graphic stuff here and so since I'm experienced I have another demo for you guys to show how actually it works so we as you can see we have two windows open here the top one is the terminal open to show how we use safe motion and the bottom one is the browser open to show our demo applications so we already have two cloud foundry instance we have our internal PCF we call it a dojo and we also have a pws so our goal here today is move our demo application running in our internal psa which is dojo to pws so let's get started first let's take a look at what we have on our internal psa which is dojo so we have a script basically the script we are target to our internal psa and do the login let's do a safe apps as you can see we have two applications one is the agent as team already talked another one is our demo applications so the demo application is basically show a bunch of albums all the albums will actually live inside the or my security database so now we are going to add a new one to show that during the migration the data actually persist so as you can see we add a new one and it's show up here then let's go back to our terminal and we already in our in our route project in the root folder of our project of our demo application and we have we create a file called multi-cloud so basically this file we are contains all the information about the binded services so in our case we only have one which is the message you for each of the service it we also has all the information about what is the service name and what is the plan name in different marketplace of different cloud foundries so in our case let's let's yeah for example my cq in pws is called a clear db the provider is clear db and the free plan is called spark that's how cloud our safe motion knows about all the services okay so let's continue let's go to our pws to see what we have there so we have a same script for that we are logging to pws and if we do a safe apps we should be able to see only one applications running which is our agent yep so let's get started so in order to use that motion we need to give him three different arguments the first one is the application name you want to use for the on the destination cloud foundry instance the second argument is the alias for the for the source cloud foundry instance which is our internal one we call it dojo and the last one is the alias for the destination cloud foundry instance which in our case is pws so first we will target to pws and we push our latest application with the latest change to pws and then it target back to dojo and then let the agent do the restoring basically grab all the data in the massacres services and put into a file and push the file to a third-party storage and then target back to pws create a new massacres service instance there and then let the agent down retrieve the backup file we just push and then restoring the restoring the data in the backup file into the service instance we just created and then after everything's been done bind the service instance into the into our demo applications at the end do some clean up and the last step is to start our application so theoretically we should be able to see our second applications in our pws environment so if you do a step apps we should be sale demo application yep and now if we go to the application we should be able to see to see the the album we just added so those this this demo basically shows how we use that motion and the data actually persisted during the migration thank you wait something not right here well I noticed that there's no low valence or like there are two domains name right yeah good catch you're totally right so so cloud foundry oh no I'm sorry set motion is a fairly new project we just work on couple weeks so we have a lot of stories in a backlog including load balancer so let me give you a number for for non state for services if you are using step motion there would be no downtime no data loose for our state for services we would be we would do the incremental backup for for other services basically as team said before we are constantly do the backup so why it come to the point we want to do the migration the difference between those two database or some other structure would be very minimal so we can either do a blue green update with minimum data loose or zero data loose cause the difference between those two is could be optimal to zero and also if we want to make sure that there's no data data loose at all we can do a red red green update and with cause the difference is small it's optimum to zero the downtime would be also based on the difference I see and in the future we actually we introduce something called a policy driven server so this server will keep track of all your cloud file to instances and then it will it will receive something we call the policy files let's say that you want to migrate your application because your cloud file to instance keep failing your application for like five time a year then you can specify the metric for this policy file to say migrate your app because if you see that it reached a limit of failure then yep so this is the policy driven server yes so that's our two like features we want to implement in the future in the late feature and also we have some other related projects from Dell MC one one of them is not school so what does not school do is basically we are replicating the whole production environment not just the data but also with other applications and we ends into a development environment so yes yes yeah no man can kill not school so so let's say there's so big bug slowly crowded into our production environment and stays there forever because a development environment is another parallel word of our production environment so there's another version of this bug crowded into a development so traditionally in order for the administrators or the developers to debug the problem the need to for the developers only allowed to look at all the logs to debug the problem it's kind of really hard to do that because they're not allowed they're not allowed to touch the production environment at all but if we have those identical developments environment so it will be very easy for the developers to debug the problem they can just jump into the development environment and then do whatever they want to to find the bug and do whatever they want to kill the bug so that's what does not school do and we are also have another project which is yeah which is called recover point so now let's travel forward in time for like 50 years when zoom in and I sell this see a motion and we became millionaires now so we are really rich and which is bad enough sure and we will have two data center under our name so you see 10 and zoom in data center and we use this recover point product to back up my day my data into his data center and because I was so rich I mean I will be so rich that I build my data center on top of a mountain because I have I want to have a good view finally I realized it's a volcano mountain so it's a it erupted and and now lava eating all my data so I have to call zoom in like can you back me up nope late and total finally I agree after long conversation so he just have to redirect the low balancer to his data center and everything is safe himself yeah so that's all we have for you guys today and we would love to take any questions yeah yeah so we did some investigation previously so one of them is to do the incremental backup not through the database but through the the file system so but you need to income you need to in control of the the backend service so you can have control of the file system so we did some incremental backup of the file system the whole file system so in that case those motor would work for every single yeah like different database what our service you want also we have we figured out like for for database like my cq or MongoDB they have they have the a way for us to do the incremental backup without the control of the backend service they have the log file the log file system and not log file mechanism so basically if you have the database access you can have some binary to generate a log file that we are basically like information about the whole database you can have this log file to create or a dynamical database so we can like do the incremental backup of the log file itself so in that case like even though we don't have the like a control of the backend service you still can be able to do the incremental backup for the for that kind of service but that is for now we found only in support for the MongoDB and the message you and some message you related database yeah is that good answer question cool yeah so I the the project is another team's project if you are interested I can give you the like information you can talk more detail about that is that okay for you yeah currently it's in our internal github yeah we are from Dell EMC so we are talked to our PM we if you are interested we can like a shell the contacts or something yeah yeah cool so that load a balancer is not like a real load balancer it's more like it's more like a traffic redirection so it's not that like smart actually like a dynamically read the load the the the traffic it's only like for the domain change say that like your application run on PCF it has like a run the IO domain but for the user that don't want to notice the the application change right the only knows that you are like actually like Dell EMC.com don't want to notice that you've migrated from this cloud foundry to another one so the map so the load balancer is actually using more like a DNS server for us in this case is that answering your question yeah thank you I'm sorry you so provide it would be if you if you are using through the second like approach as that like just have the act doesn't need to have the back end service it will just like as the normal one just copy the data so it will copy another one yes yes yeah that's the load balancer one like we want to use load balancer as the way to route your stuff yeah okay all right I think that's it thank you guys thank you for coming