 Hello? Hello? Okay, it works. Okay. Hello, everyone. Thank you all for coming here. And I'm Huicheng from Senang Corporation. I'm also a community manager for China Obstacle User Group. Today, my topic is DevOps in Obstacle Public Cloud. So, what's DevOps and what does it look like for Obstacle Public Cloud? And DevOps is described in Wikipedia as a software method that combines the communication and integrated of software developers and operators. But here, today, I'm not going to talk more about the development methodology of DevOps, but instead, I want just to answer some questions. For example, what we have done in the past years for Obstacle and the world, how do we operate Obstacle Public Cloud? And so, what does Obstacle Public Cloud look like? Here are some examples. HP Cloud is the first and also the biggest Obstacle Public Cloud. It's under 90s. It was totally based on Obstacle. And Rex-based Cloud and the Internet will again key in the United States, in the cloud in European and the KT from Korea. However, they never tell you how to operate it. There are public cloud based on Obstacle and they never... But since then, as an Internet company in China, we operate the first public cloud in China and we would like to share with you about how we implement our own public cloud. I hope you... I will let you down today. I have several topics. The first is about changes in building Obstacle Public Cloud, including network technology and security enhancement solution and identity integration, viewing and monitoring and dashboard improvement. The second part is talking about how to operate production in Obstacle environment, including platform connection, automatic development and CI system. So there are so many topics here. So... Also, the third part is talking about the second part is community Obstacle Public Cloud, which is initiated by Sinan Obstacle team. Sinan is... Before I go to... Before I move forward, I will talk about our cloud requirement. Sinan is an online media company for China and the Chinese community around the world. Sinan was founded in 1998 when the World Wide Web was just introduced in China. It now operates several multiple biggest websites in China for tens of billions page views per day. Now Sinan operates two major business. The first is Sinan.com and the second is Sinan Weibo. Sinan.com is the largest mobile information department of a portal in China. It provides various online services like news, finance, video, email service, and blog hosting. For Sinan.com, it is a unified infrastructure and app platform to host heterogeneous apps in one single platform. So as for Sinan Weibo, Sinan Weibo is a micro-brown service and more often is regarded as a hybrid of Facebook and Twitter. Weibo was launched in 2009. It has experienced explosive growth in its early days. Now it has 15 million users. It's definitely the number one SNS social platform in China. And Weibo also has huge influence on China's society because it speeds up the information flow and protects freedom of speech. Weibo also has an open platform. It wants to build a social ecosystem through its open API and it's called environmental. I'm from Sinan. Our responsibility is to build a reliable, highly scalable and secure cloud infrastructure to support our own business as well as some external customers especially for small companies and startups. Sinan Cloud has three different products. The first is Sinan App Engine. Sinan App Engine is the first and most popular PS platform in China which launched in 2009 supported web application written by PHP, Python and Java and now it has 250,000 developers who have developed 380,000 apps running on SE which is a big business in China. The second is Sinan Cloud Market is the first cloud market in China. It's a cloud store based on SE technology which means it can lower the price to a very low level. It was designed Sinan App Engine is based on web developers while Sinan Cloud Market is focused on ordinary users. Sinan Web Services is the first cloud-based company as cloud in China. The rest of our slides we'll talk about this platform. First, I want to talk about our involvement in the community. From the beginning when we began our project, we were deeply involved in the community development. In the first release, we are ranked in the top 10 list by change set or by bug fix. So let's talk about the challenges to build OpenStack public cloud. First, in our experience to deploy in OpenStack environment, the biggest and possibly the most important issue is not working. Here is a comparison table of no network and the quantum. Before ESX, we have only one choice that is no network, but now first we have another turner key with quantum. So which is better for you and which is better for public cloud? We can say that quantum supported turning and ESX, but unfortunately it doesn't support a multi-host and a secret group until now, maybe we will support after the next six months. But for now, it will be some issues to maybe some performance issues and security issues in public cloud. So here is our comments regarding the choice. No network is simple, robust and reliable service even though there are some advanced features, which these features may be not very applicable by most of the adopters. So this is not ready for production use from our experience and from our viewpoint, but it's always has a great potential to be the over-sourced MVP solution. So my suggestion is to continue using no network in production environment and clear the next release while paying a close attention because you can now quantum have OS and open source and other solutions or the commercial backend such as Nasera backend and Cisco uss. And so we talk more about no network things is is running very very in our production environment so I will talk more about it. No one has three different network topologies first is FNAT. FNAT mode is not very practical deployment since there is some external ACP server and some human intervention it may be not very flexible. While FNAT ESCP in my viewpoint I think is amazing EC2 networking even though I don't know about the underlying infrastructure of amazing EC2 but from a user point we can see that FNAT ESCP and amazing networking share a lot of common features and quantum in this never more like amazing VPC FNAT ESCP is basically what can exist the machine and get an IP from one single network which is very simple and easy to hack now FNAT ESCP is widely used in public cloud and also preferred topology in many scenarios both for private and public cloud. VNAT manager is a new company from our viewpoint it needs some hardware configuration I'm not very suggest to use VNAT manager except you have very strong isolation here is here is a rear a little topology in our production environment I need a different from the official recommendation because we have done some customization first this topology this topology works very well in our production environment but it has some drawbacks the most drawbacks is that all the virtual machines in the same L2 layer it will be very hard for in large the scale of the virtual machines so it may be a potential issue in this network but in our production environment it supports hundreds of features it will be okay in our production environment and I should for this diagram I should write a very non-article to articulate our configuration and our tradeoff to design such a topology but I first talked about some basics from the pictures we can see that it has three different networks red lines represent the virtual machine and black represents the management network you may notice that virtual machine switch and management switch has the same uplink to internal core router there may be dangerous for us because in the opposite environment there will be a lot of external users which is not trusted how to handle these securities we have done some security enhancement we have developed a service theater extension to secure group in our network it is used to filter the egress traffic from virtual machine to our internal network from outside to to the virtual machine the service theater and secure group they have a reverse direction theater so service theater is to define the users whose traffic could be able to reach internal network IPs and segments so for the storage solution storage solution may be the next biggest challenge to operating a public cloud normally we have two types of storage one is object storage amazing S3 now thanks to SWIFT we can very easily build object storage system using SWIFT or PONOG storage and can now see there is in our viewpoint I think it's not the equipment of amazing EBS even though many developers and many guys think it is amazing is over sourced amazing EBS but I don't think so it's just a framework to include multiple over sourced and commercial solution well no one warn you and Asuka's cinder is also not applicable to public cloud because it's storage IO traffic is totally based on network but the design their architecture is not distributed and redundant so they maybe have some problems in our production environment it can not be tolerant for us so we don't use no warning to Asuka's cinder but for SHIPDog can also serve I think it needs some time to be very mature maybe one to three years ago then you can consider this plug-in I just suggest you to keep close attention to this plug-in so we development our own solution that is the INAND this is new plug-in or a new cinder storage implementation it has such features which is high performance storage and it also supports incremental and independent snapshot which this snapshot has been pushed to suite for persistent I have proposed a topic for design summit tomorrow and I will discuss more about the implementation and the architecture of this plug-in for identifying integration for Keystone as you know AWS pride is a uniform dashboard to enable users to connect all regions around the world this ability depends on powerful identify and access management system in all stack Keystone has the same ability I think and it can provide the same function for us here is an example how we use Keystone to support multi regions just like AWS for Beijing I have a region Shanghai we have a region each region have keystone instance where the keystone instance share the same database the same that may be connected to the same IP or product or the two database are replicated both okay for monitoring there are so many monitoring tools in the market like Ganglia, Nagyos and so on all could be used here but most of them require to give noise into machines that is not a tournament for us because many users may not trust the internal process so we implement a simple framework to for monitoring just some agents in the different hypervisors and connecting the CPU memory and network traffic for each instance and then aggregate this state to the non-seq storage and the dashboard access this and aggregate this state through API and we also the implementation in GitHub where another incubated object project named a thermometer it also address this problem the reason we didn't use it because we implement our own monitoring system before ASICS where it's not proposed and it's not incubated project now but now I suggest you to consider thermometer because it's more universe our implementation may have some specific consideration for our own environment here is the dashboard monitoring for BnM BnM system is about how to make money so it's definitely very important component for public cloud and obviously the signal meter also address the BnM problem but in our implementation we separated to different project the features including keeping track of BnM keeping track of BnM information to charge tenants is very flexible customization or payment policies it addresses how much and how often to charge for resource unit it handles some BnM method such as paid or pay as you go it also has coupon coupon support you can also check out the source code in the GitHub we charge for instance bandwidth and public IP and node balance first let's talk about the dashboard first I must admit that we finally abandoned the official dashboard project namely Horizon the reason is very easy it's not very easy to customize it's not very easy to customize so you can see in the cloud market in order for best cloud they are all using the same identical dashboard so we don't like the official UI so we need to customize another technical reason is that the backend implementation is not very convenient for us to do second development what we need to do is to customize another reason is that it's very hard to keep up the space of Horizon or you can see Horizon cannot catch up our own requirement so we decoupled the front-end design the backend implementation to make the dashboard at night wait front-end the second thing we separated the user console and the admin console we have pay another mention attention to admin console because for us admin console is very quick so we need to customize the core cluster the core of the environment so this is the official dashboard I can show our own implementation so from the beginning we have the first version we totally change the UI but also based on Horizon and then the second version I need to like AWS but then it's also based on Horizon but our latest dashboard is totally abandoned of Horizon divided by ourselves so I will talk more about this some new features in our dashboard you can see the different square means virtual machine and longer encounter means there may be some performance issue or some risk in this so you can see what happened what's the problem and what's the issue within the virtual machine and this all of you use the dashboard for monitoring very like AWS monitoring page for then for okay this is the admin console for admin console in the left side we have another small square each small square represents a physical server and the number within the square it means the number of virtual machines in this host then we can you can you can see the detail detail status of the host this is another view of admin dashboard so that's coming to another, the second topic how to operate production of stack environment first we use to connect different which server we use the OS distribution the hyperweather the storage so for the physical server we use because the stack is very independent for the physical server so it can we can keep an eye on the stack so we can keep an eye on the stack in our existing physical servers so what do we have is 2-year X86 and we use Ubuntu because Ubuntu is in the beginning and the more important is some scripts and has only work in Ubuntu so another reason is that all our stuff is using Ubuntu desktop we are very familiar with the package system with the software management and for the hyperweather we use QM so in general we hardly dispute about the selection of QM or then because until now QM already run but also from our own viewpoint I think QM has run the battle for the hyperweather selection until now we can hardly heard that some new project then most of them are QM the reason is very easy QM is more easy to manage we do not need to invest much on the hyperweather layer we just focus on our overstate service development and focus on our deployment we no longer care about the hyperweather because QM handled all the things for us for the underlying storage for some physical servers we use SAS disk with red 1.0 or red 5 support or for some other type we are using SSD backend which some of the customers need high performance high performance high performance sorry for example for MySQL database and other allocation so the challenge is give annoying cloud cloud is big data center especially for public cloud so the requirement is to provision large scale physical infrastructure in very few minutes as well as software development most importantly we need the tourists to orchestrate all the heterogeneous components including the physical servers network devices and other components which is a very big challenge for us here are some automation tours we use our production environment the never side is development tours we use GATE for the code hosting and for the word management we also use a DB repo to build the package for testing and for production environment and for the operation tours we use the most two important I think the most important tour is FOMEN FOMEN is used to give an OS on the bare matter automatically and PAPID is used to deploy a service or to do system configuration upon the operating system so when the OpenStack class is ready we need some other tours we use Xabix to monitoring the physical servers we also have our own implementation our own dashboard to monitoring servers and at the same time we use some open source tours here is our CI system we use Garrett to do code review and the Jackins was triggered by another events to do the packaging to tell the users your commitments fail or your commitments success and I think this core CI system is totally from the communication it's from the community why we use these tours because the community use it so we use it in our for our team collaboration the whole system it works very well in our employment for project management we use open source variant notepad we communicate notepad in in-house and put all our internal project to our in-house notepad so the second thing I will talk about is community public cloud for us so one sentence to describe is a community free public cloud more than just open-source sandbox I mean before there is a tri-stake project which is just open-source sandbox and maintained by the community but for Stagnav it may have some different mission so I will talk more about it another reason why we notice Stagnav is that tri-stake is not accessible from China because it's using the Facebook notepad which is totally denoted by the government due to some I don't know the underlying reason so okay and why Stagnav not everyone has the ability to run an open-source public cloud when no resource is no users, no market but Stagnav will change this what does it mean for all for many of us we most of the developers do development in-house or build a tri-system or just build a a delta-stake instance in the laptop or in one single physical service but most importantly the developers cannot know the re-experience of users no and the developers also didn't have the given opportunity to operate online open-source environment it often means another for investment even if you have physical service or you have resources you can't get a lot of users to try your platform so Stagnav has three goals for different users for different layers for obviously users who experience Stagnav they understand what exactly Stagnav is and what does it provide for users it can also have applications on Stagnav or use in Stagnav API which can build a phase of Stagnav this user can be the later adopters of Stagnav or a support and for OpenStack computers or developers involved in Stagnav they can test in batches on real production like environmental and get feedback and they can also facilitate the development and QA process and also gain the experience from operating a Stagnav without the risk of SRA because it is free and the developers also can have better understanding the requirement of OpenStack users or their potential customers for Stagnav sponsors and in OpenStack community and also can own one Stagnav region in their own data centers they can also have free access to our technical support and consultant from Stagnav DevOps team I want to emphasize one point that is for Stagnav sponsors who sponsor the physical service they can deploy the one Stagnav node in their own data center which means that you own all the resources and you can get the users from outside and you also have the support from the developer and from the community so how to join Stagnav for is very easy just go to Stagnav it's already online you can access it and and without improvement by the mean and for OpenStack contributors to join Stagnav they can first must pass away their company to become a Stagnav sponsor thus you will have Stagnav region in your own data center you will have the administrator of the Stagnav for OpenStack sponsors just sign agreement with Stagnav and Stagnav have you to build a Stagnav environment and then your region is listed in the Stagnav portal so you will get a lot of users from the from the company there are some ways to join Stagnav and this is the dashboard for Stagnav you can choose different regions before you log in to the Stagnav after you log in to the Stagnav so I have a summary first OpenStack is definitely the best open source cloud platform for building public cloud I call it a solution because I have tried and compared OpenStack from other open source cloud platforms just because our sincere trust for OpenStack this trust comes from in particular for open open source open community and open development and open design now OpenStack growing first growing ecosystem around OpenStack is no window-locking features also provide us great opportunities not only in business but also in career another is that OpenStack public cloud is much more customization and development to differentiate you from other OpenStack public cloud and also requires strong technical skills and development in the community okay, thank you I'll check the time maybe the time is so if you have any questions you can ask me later