 Thank you very much for giving an opportunity to make a presentation in a top-notch conference and Thank you very much for coming here. I am a Hiroya Soyama and I came from Japan and I want to talk about information management and Please don't be say about myself and our company. I've been a software engineer and I'm a former OpenStack Technical Contributor even though just only a few commits and now I've developing and maintaining our internal information management system for seven years and I'm also a contributor of StuckStorm that is an automation DevOps open source software and About our company DMM.com it mainly provides entertainment contents for domestic customers and It engaged in a large variety of businesses on many areas. We are focusing on online video streaming services and The total number of customers is over 39 million people and the last year sales sales is over 305 billion yen. That is about 2.2 billion dollars and We operated those services using our own premise environment This is summary of our infrastructure. Much traffic is a feature of our environment compared to the middle scale of nodes And that's my short introduction. So let's get in the point I'll show you how we have managed our infrastructure information and what we struggled with This story is composed of three parts at the beginning I'll show you how we have managed our information and what problems were happened and then I want to talk about an ideal way to manage information in response to that problems That way is SSOT and that sounds for a single source of truth in conclusion But SSOT is not perfect solution. I mean the flawless solution doesn't exist in the world But trade-off does so I'll mention shortcomings of SSOT. Lastly So the first topic I want to talk about our past thing This list part of the information to operate our entire infrastructure basically We have used VMA solution to manage both physical and virtual machine storage and network For production environment. I'm sorry for not to use the open stack but Our entire infrastructure operation is not complete with the VMA alone It includes not only Deploying and monitoring but also the purchasing and delivering and repair control and also accounting also on and It requires to grab any kind of information to relate it to them and We used the rack tables which is open source software for data center set management system to handle part of that information Our rack table was a good solution to manage physical appliances like computer nodes network switches Fireball was such a dad and They are port connection topology, but it wasn't good enough to manage that whole information So we supplemently used another way That is one of the most famous Information management system around the world that is Spread the sheet. Ah Thank you very much. I respected that reaction But it may sound ironic, but I don't want to say it badly. I'd rather say that it stunning and fabulous for User interface and also the functionality and it grabs the heart of tons of people tightly But however, speaking of managing managing a large variety of information We should use alternative. I believe So I want to describe what kind of problems were happened by them The first problem is spurning other data sources that have a duplicated information with others At that time We had managed infrastructure information such as which server is equipped on a rack and where that rock is installed on a floor using rack tables and Its information was opened into public. So Everybody could access to it Once Nastaf started to think that hmm I want to manage their authentication information like password was such as that for each appliances But I don't want to store it to a public place. So let's make a spreadsheet that has confidential information. I Think many people often do that to separate confidential information from public one and This story doesn't end here another team staff that is referred to only this public data source also think about the same issue and Created another data source that has almost the same same danger in a different way It may seems like a joke But it's a serious and it's not laughing matter because they might be inconsistent in the worst case both of them a check and Check each item is under correct them one by one And This is a big loss of course information management came to be the work to keep it consistent But manage me about managing multiple data source makes its cost double or more if another data see another data source is Existed Furthermore, even though you work this task as hard as you can The product value will to be better and it will to contribute to increase customer satisfaction again to be clear I Believe this task is necessary for information management, but it's good enough to do it just for one time So we were suffered from those problems in summary information management without policy a result in its propagation Spurning data source one after another and Those multiple data source require as much cost and it doesn't contribute to increase product value and the customer satisfaction After here, I explained the hardships of information management From now, I tell you how we tackled that problems It's a rough conceptual diagram of ideal information management way for us This means that data sources are gathered at one place and that means SSOT a single source of truth And this idea is very very simple and it's easy to achieve it by gathering scattered information at one place But it also easy to break apart because as I said before Information data source flew around like a dust if you leave as it is So you have to make a mechanism to bond them into a system That is a critical point of SSOT and to realize it data extensibility is quite necessary and It must be able to do it dynamically because information grows up day by day and This kind of information tends to increase but settle into decreasing and Access control is quite important for SSOT I need a feature that part of them are visible from a specific user who is belong to TMA in this case But some others like accounting information to be invisible from that user Ultimately, I believe that only two principles are sufficient for an information management system to become SSOT First is dynamic extensibility and the second is the proper access control Of course, now information management system needs various other functions like full-tech research and following changing history or such as that But these two principles are essential for solving the problems that I mentioned Throughout our discussion, we decided to develop an information management system to achieve it on our own That is A1 and that is published as open source software in 2020 These are main concept of it I believe this is not a software that human being have never been invented But I could solve it could solve the problems that I said before The reason why I show now very ordinary software in here That's because I want to describe it as there are basis for discussion If someone suggests a better solution in response to my presentation it must be good for audience and for me too and I'm also happy when I can contribute to give you even a hint to easier problem related with information management in a way and So I'll explain the concept of this software This has two main data structure that handle all the information that is model and entry The model describes data schemer Which means what type of data it has that is like now table of our DBMS and relational database management system like My scale SQLite or such of that Model has zero or more attribute that has type of value to accommodate and The second main data structure is entry And that describes instance entry consistent attribute that correspond to models one each models of Each attribute of entry has actual value based on its type These are like record and the field of our DBMS that's our own data structure concept and This is an example of model declaration and each gray square and Here is it. I'm sure. Ah, yes here here is a model and blue square that is in each model here here and here are Attribute and the data type of attribute is very primitive like string reference Boolean or and its array or dictionary and then you can make the data structure like the that represent on-premise environment and This shows entries and that corresponds to each models each blue rectangles those blue rectangles represent entry and orange line Describes what entry refers to by reference type attribute in each entries From this data the user can know that the server one refers to Submarine is a gift on rock one or two and rock one or two is located installing a floor one or one Three floor Once we thank you and this floor is located on the Tokyo site or such like that and the point is in this way You can handle hold it all better like Directed graph that is quite easy to handle from computer program And to describe how a system the principle are achieved in this system I'm take a closer look at this entry over subarack and in this case and Rock one or two entry has attribute to know which floor is installed amount of power supply amount of power supply and whether it is available or not and After that suppose you want to add lower capacity information is needed for the rock entry And then you can do it by changing its model to add an attribute like this and You can set the actual value on it. That's all you can expand the information dynamically in this manner and Speaking of access control It can be said for each model entry and attribute levels in this case team B member can access all the information that is stored at rock one or two and You can set on config a set is configuration to prohibit to access to some attributes from a specific role user and You can also prohibit to access to whole entry from others By using this role-based access control feature our back feature user don't have to create another data source, but just to add attribute or model and Set access control configuration This shows how many models its attributes and entries are increased compared with the first release as you can see both the type and the amount of information handled by air one increase the the path of three three and a half years and after releasing air one internally in November in 2019 people no longer create other data sources instead they store information to air one and Expand its model as needed That was this and that was our story leading to the SS OT From now, I'll show you how we make them make it the most of The first use case is application deployment This deploy a deployment system uses staxome That is famous for an if workflow engine At the beginning of deployment it gets source code from a github repository and before deploying it To each notes it access to air one to get information where and how to do it This design is able to divide policy from mechanism to explain that I am taking a closer look at them Air one has configuration information. What knows make up which notice it and For example, this attribute Okay lost Here yes Examples represent which nodes consist on a specific notice it to deploy and a staxome we can know Where there are three nodes on it then a staxome deploy that that Nose according to workflow and Staxome also get information how to deploy them as such as environmental variables of containers and passing parameters for work for In this way It decouples policy from mechanism It makes them as it makes them simple and increase the flexibility and the maintainability because if the policy and the mechanism are combined You have to reproduce CICD processing whenever you change its configuration or setting information in addition it also decreasing the Developing cost by being an SSOT it means if there are multiple data sources like this You have to implement features to get information by learning what kind of API endpoint is and What response format is or what parameters are For each data sources, so however if the data data management method is unified you don't have to learn it Again once you did and you can do it as same manner with other data situation So the next yes That's our one case and the next use case is for data center management information management And we operate on-premise data center and its information is managed by air one and this is our clipping picture of it it shows some server racks on a specific floor and Installation status of equipment and each rectangles each Black rectangles represent the server rack at a 19 inch and 42 unit regular server rack and the blue roof tongue The blue rectangles is the each racks That shows Appliance and that have already been set on it and the compulsory the white is empty and red represent the reserve of space So you can see the Sorry, I can see it. So the you can see the fight. Yes, okay The also that rug is almost empty and in the middle line maybe that is reserved for an airflow was such a dad and If you click on a blue line, then you can see the information that is equipped in this space So this feature is developed by requirements from the customer But this is an exceptional mechanism of air one As I said before all our information is composed by entry and all entry has on you are L You are and end point to respond its information For example, once user access to the rock one or two entries and the point like this Then everyone responds the information only associated with its entry like this so basically Air one responds information for one entry power one access So this principle is necessary to be a general purpose information management system However, this basic mechanism is hard to show this page This is a creeping feature shown before Because this includes each rock entries on a floor and also appliances entries in each rocks It shows them all at once So this this is against the one access and one entry principle So we made another mechanism to achieve it This enables to customize the view processing according to the structure of a user defined model. We can call it custom view By using this user can get information from multiple entries when user accesses to an end point that custom views is configured in this case when user accesses to the Floor one or one entries and a floor one one entries and the point This custom view will get information from its entry and also Suburqs entries that refers to it It could be identified by pulling their reference graph data structure in internally In this way user can get Overview view page like this At the time users realize that the more information is gathered at SSOT The more it will be useful by connecting related information using the custom view feature before long many other custom views like IP address management and a port physical port and its connection management features were created and Users are actually added their own information to the air one and also expanded models Since then SSOT movement was accelerated Before that we were struggled with information disposal and a spanning and a spanning around But now it feel like the information is gravitated to the air one on its own That was our story that was our SSOT story From now, I want to see other companies activity from public information And that seems to be related with SSOT the one is Roboton which is published by Meta Facebook It managed a network life cycle from the from design configuration deployment and monitoring Those the status is managed by SSOT and data SSOD data source, which is called FV net And another one is malt by Google It's a large-scale network management technology from intercontinental communications to data centers It doesn't mentioned about SSOT itself, but it seems to work using a single single topology database Like these examples, managing information in a single source brings benefits to infrastructure operation, I think Until here, I talked about only the positive aspect of SSOT But according to the sage, there is no silver plate So I will mention about negative aspects of SSOT on the basis of our experience at last One day I received a claim from our operational team staff, hey I can't update our own VM entry and this system must be wrong, he said Until then we managed virtual machine information by this model But at the moment, accounting team staff changed its model to set status attributes as mandatory one Because they have been worried about the virtual machine entries of empty status attribute They can't know whether it's actually used and deserved with bidding target or not So to prevent to its omission, the staff decided to set it to prohibit to update and create its entry without setting status attribute This decision is very reasonable and this information should be managed like this But this stuff should understand that all the information that is managed in the SSOT is shared with others So it's necessary to coordinate with other users about changing its model in the SSOT environment So if you ask me that, okay, the model shouldn't be changed, right I'd say neither Because sometimes it should be better to actively change models I'll show the case of it The right side is a model structure related with network appliance and the left side shows its entry That are initially registered That model was registered designed to manage information that is just equipped on a server rack and Entries are only assumed for network switch and routers at a time But before long, a large variety of appliances information started to be registered at Airwan like physical and virtual load balancer and firewall And then it also needed to save other information such as HCL policy for firewall or Virtual server information for load balancer But those information are irrelevant for each other If these attributes are added in each model, it will be more complicated like this They are color-coded here and for convenience, but it must be difficult to tell whether whether ones are actually used and which ones are not at a glance So and those attributes are never been used by original network appliances and Adding attribute without maintaining and coordination complicates model and increase each its management cost In this case instead of adding a various attribute to the model It's better to divide models for each purposes like this Then each of them each of their maintainability and could be increased This situation gave me a lesson that initially defined a model and its purpose must be changed by growing number of information and Because information is naturally increased and unexpectedly Now we have to keep organizing and maintaining it not to not to get it messy and that's all I've introduced about the information management experience and SSOT And speaking of the best way to manage information I don't know the best way and I don't think there is not such a thing Because all individual solutions have trade-off But as far as if infrastructure information management is concerned I dare to say that SSOT is the best way even considering its disadvantages Because it could not only increase flexibility and maintainability for infrastructure operation and information but also decrease operational operation and software developing cost and please tell me if you have any ideas for information management and If you want to make a discussion about about it I'm happy to do that and please feel free to ask me I'd go anywhere wherever you want. Thank you. Thank you very much Do you have any questions and I ask you in a favor please If you never mind, please push the star button of this link During some question will be happened. Oh, thank you. Thank you very much Please never mind. Please tell me the name of the software. I see. Ah, thank you very much. Okay Is it open source software? I see. Thank you. Thank you very much. Okay. Is it okay to finish up?