 Oke, saya akan menjelaskan tentang Masif Multinox PostgreSQL BDR versi di sistem retail. Dan kemudian ini adalah quite a big retail, it's hardware market. Jika salah satu of you wanted to ask something, just scratch your hand, so we will discuss right away. Oke, my voice is good up to the back. Oke, thank you. Little bit. I cannot speak more higher than this one. Can you use the mic? Is this recording? Oke, I will speak more louder. So, I am Julian Dossetanda, I am presenting Equinix business solution, it is an IT solution provider in Indonesia. We are doing very much in PostgreSQL services and my company motto is an open source and open mind company. We should be open mind to use open source because very hard to approaching a client to use open source. So many, many factors that we have already discussed. We deliver peace of mind. Our mostly client is the banks, payment gateway, retail system, retail companies. You can see in the screen, Manili Bank, BGB Bank is a big banks in Indonesia and ADI. Denamon, Adira is a payment gateway and many, many more. So, one of my clients I will representing, I didn't said which one. So, one of them is a retail company. This is also our clients, Exxon Mobil, Slambusi, Alvaya, not all of them using PostgreSQL. We sell so many other products, one of them is PostgreSQL. Oke, this about a bit about our company. So, we will tell about the background. The background is the big retail system has required about how to collect the transaction, the sales from the POS system all around Indonesia. We have about more than 3,400 POS. One store maybe can up to 60 POS. So, it is a big challenge to collect the data real time in real time manner. So, before the data collection will be done every day. So, it is not quite satisfied. We should change the system to make it, the data can be collected in real time manner. Maybe every minutes. And also the distribution, the new item. Somehow, there are new item, new product, or maybe new promotion, or maybe new discount should be able to be distributed in a faster way. Before it is require maybe one or two days. Because we have 3,400 POS, it takes time. So, by using later solution, we will able to distribute the new product, new promotion right away in seconds about some less than a minute. Oke, and the last capability that should have is POS can have the capability to stay in service whether or not the network is good or not. So, should be able to stand alone. Oke, this is the problem before. The land is very bad. So, even it is a local network, the network is intermittent. You know the store has many foods and many mice. So, and the one sometimes not really good, infrastructure in Indonesia. So, we should have a system that can network problem prove that able to run without any hassle even the network not really good. That's the aim. So, this is the challenge. So, there are 200 virtual, 200 servers, 100 is real server, another 100 is virtual server, and one head always. And then 3,400 store, POS machines. Oke, that's all the challenge. So, how we solve the problem. We use, of course, BDR. But before BDR, we use the, we change the POS. Before POS using maybe one of you that already knew about Node.js, level DB, anything, it's a startup technology. It's not good for the POS, because before they use level DB. Level DB is quite heavy. Maybe POS needs database, but in fact it doesn't need actually. Just flat file database is good. Because the POS actually doesn't do transaction in terms of database transaction. POS only did single transaction. So, doesn't need currency. So, doesn't need database. Only flat file. If you use flat file, then you gain something more. If you want to update the flat file, you just send the difference, and you pass it. This is what we did. Before the former vendor update the file about some 10 megabytes every night. So, the POS should be run up and running. After we change, only using the difference, delta of difference, only couple of seconds. So, we prepare the database in the store. Actually install, because from the store, replicate the originally with the head office. And then after the store, we create the flat file for the product promo anything. And this flat file is keep in there as the historical data. And every changes, we will make a diff. And we got the difference file depend to every terminals. Automatically. So, the solution, we use the real time. It is synchronization mean we use the PDR from database in the store to head office. And then, that's the solution, and the flat file, PDR. This is what we are talking, just talking about the terminal, just using the file, not using the any kind of database. Not even connecting to the server, they are running standalone. So, when they have a problem, for example, the ethernet meeting Biden by mice, that would be a problem. And this is in the store level. So, we have one server, containing one database, PostgreSQL, PDR, and connecting to many POS. And this server, we already implement about, actually we are ourselves implement in one or maybe 10 stores. And when we come to the client, again, we saw about 200. 200 of PDR. PDR is maintained by second quadrant. It is a multi-master replication. Why we need multi-master replication? Because we need both ways, from the head office to the store and then from store to the head office. So, we cannot rely on single replication. We use multi-master replication. So, from the head office, we update the promotion, the product, the price, the store, we update the sellers. So, these sellers from the store, from the terminal, goes to the head office every second. So, the CEO, the owner of the company, can see the sellers every minute in their cell phone. That's the beauty of the technology. I will show you the picture later. Oke. So, PDR is the solution to maintaining the infrastructure from the store to the head office using multi-master replication. PDR using physiological replication. So, if you want to see the production level of multi-master replication, this is it. Even to discuss, maybe we can discuss. So, PDR become the infrastructure. It is 200 more clusters. So, one store and one database in the S-Office, it's one cluster. So, there are 200 stores. So, there are 200 databases in the head office, in single instance. Maybe we can use single instance or two instance. So, there are 200 databases exactly the same with 200 databases in the store. So, there are 200 clusters. Oke. It's all across Indonesia. This is the most important thing that I am giving talking in this event. How we manage multi-master replication in the production level using the real-time system all around Indonesia. It running right now. 200 databases and giving you the transactions. I mean the seller's transaction report to the head office every time. So, the beauty of PDR is you can PDR is quite what is it, network problem proof. So, if there are problem of networks, I will show you like this. We have the online monitoring system. So, when the network problem, there are black in the box, in the widget, means there is no connection. When is the connection backlight again become green and also the database recognize. We have tested a couple of hours but maybe up to if I am not mistaken, 12 hours there is no connection. After the connection backlight again, it will continue. So, quite tough. So, this monitor give you the network check, healthy check and also give you the status of the updates because it is too small. So, there is a number of store and then the has, this is the updates of the file from head office, sorry, from store to the, it's terminal. This is also the monitoring system of the application. The application give you the data of status of the healthy of the system. So, this one, you can see the price here, means the onset of each store. I cannot show it to you, so I have to make it black like that. So, actually we can do without multimaster application. We can create web services. We can make real-time web services, not so ever. But it takes a lot of effort to maintain also because web services we should able to monitor whether it's run or not, synchronous or not, etc. By using the BDR you gain something more better because it's already done in the BDR. What you should do is to check, or to monitor the application status. Any question? You have no conflict in dates that are right? Yes, the beauty of this implementation we access different tables so there's no potential of deadlocks. So, this one is following the distributed databases. Multimaster databases actually intended to best implementation is for distributed databases. So, there's no potential conflicts. When at office update the data, they use this table and their table the sellers use another table so there's no potential conflict. Any other? Yes? Okay. Replication of Multimaster Replication using sequence. So, what we need to do is to check the sequence. If the sequence is running well so they are also in PG status then it's run well. So, in PG status there is a flag the flag and the sequence is the two things that we check to ensure the status of application is in a good way, in a good condition. Yes? We don't have any others. Do you like to see how much the sync is done and how much more the sync is done? Ya. Anyway, we don't have it. We don't have it. I think you can check that from the logical replication flow. Maybe you can see whether the how many difference it is from the head office to the stores. But for us, I think because replication should be have the same status, right? That we check that we monitor is the sequence. Whether the sequence is the same between in the head office and in the store. We also did some second check with using the query. We sometimes doing something like count the table. Ya. That is no longer. Like, do you have any kind of a data which says that you took this much time to sync my database from no data, no read? Ya. So much of transaction where sync of kind of facts that you can share. Ya, ya, facts. Ya. In bug, bug updates. So that become problem. Because the network only 1 megabytes megabits per store. So when you send the huge amount of data from head office to the store it will lock the connection and eventually database also not running stable. In some cases in some cases when maybe the partition is broken or maybe synchronization is broken we can do something like re-synchronize. We build the nodes. So one cluster consist of nodes. There are master nodes and there is child nodes. So we rebuild the child nodes to re-synchronize to the master. That's sometimes that we should do. Yes. In that scenario that one table is in sync and the other table is not in sync. Let's say I have a table D and table B. So the synchronization is done at the database level. At the database level. But let's say that there is a scenario where one of the tables is not in sync. So is there a way we can detect that for monitoring at the database level? I'm not sure we can detect which table is synced or not synced because the replication is done at the database level. So as long as the synchronization has the same sequence it is done. But we cannot determine whether the table is in sync and another table is not in sync. Ya. We don't have that kind of check per table basis. We check per database basis. Ya. Any other questions? I have a few of the monitoring. So the monitoring actually is simple application. So why this monitoring can be a good monitoring because show you real time data. How they can show you real time data because of BDR. So because BDR the beauty of BDR so we have no hassle in communication data from head office to the store. The store in Indonesia is many cities and also some of the cities in the very far way to the sea. So having actually we have not really good network communications. So that's the fact that we face but using BDR we we can what is it? Use the BDR on top of that kind of infrastructure. So there's another what is it? the warning or maybe downside of the BDR. Not all object is replicated. It's sequence and non deterministic function any non database object. So for all you should create input databases. So the conclusion is BDR is a good solution for the retail system with has multiple or many stores it can reduce the overhead in data communication data synchronization and data integrity. BDR with its logical replication complete both way using the multi master topology it ensure the data availability and has strong tolerance to the network latency and intermittent network problem. So any question? Ya? Sorry? 200 databases 200 nodes How many nodes do you have in your master system? 200 200 database or one instance We can back up the instance, right? We should fell off of all of them. Ya. And is there a scenario where those in the stores try to update a table in the master simultaneously? So is there a scenario where there can be a conflict between the masters updating the same table at the same time? Conflict between two masters? Ya. Because this master have a different side with another master I don't think there is any potential I think. I think you have two nodes at the stores try to update a table simultaneously at the same time? No, there is no a way because physiological done in a logical database so there is no potential conflict You take the conflict resolution like say that there is a bi-directional replication No, we don't take care of the conflict resolution because it is already done in BDR BDR has a single connection for a single database So there is no In the postgres in the postgres criminology about databases single databases is independent than the others So BDR run in the database based communications So there is no conflict there is no potential conflict Any other questions? Yes Yes Oh ya Ya, actually the POS system running using the flat file and creating flat file also transaction one file and you can like for example you have a struc and you buy groceries and that struc is one file so if there is a this problem in network get offline so there are many files inside the POS actually and then there is agent inside the POS and going trying trying to connect to the servers every time and when the servers up again or maybe the network become online so the file is sending to the network servers automatically right way after that from the struc the file is inserted into the databases okay that's also automatically even basis and then the database from the struc will automatically replicate to the SOVS and then the database in the interface automatically start with the SOVS so we can show in the monitoring system real time so the owner of this company can see the transaction, the offset the sellers every time real time using his mobile sorry to the stores what is the the question is there any changes the data is not synced the data is not synced, there is a possibility in some of the steps for example the BDR and then from the store to the POS but we have a check every time in every point for example BDR will ensure you the data is synchronized when the sequence is the same there is a capability of that so when the BDR condition is healthy so you can ensure this will be good okay and then from the store from the terminal to the store it's automatic something like copy file we use SSH and we can check how many file in the POS and how many file in the store and we send to the SOV we have herbits in every point in terminal to store there is herbits store to the SOV there is herbit and this herbit send the status so the status goes to the monitoring system so we can check through this monitoring system whether the connection is good or the data is good synchronized or not we check yes why not use BDR from the POS no, we don't use database at all in POS we use BDR because this is quite your own flat file synchronization routine now POS is only 2 process of atom process of it's very small formula POS is very small POS so we don't really need this hardware no no POS cannot handle browser we use browser-based application cannot handle the Node.js server and BDR all together but if you use only the browser, Firefox and Node.js server in a flat file is very, very light no problem so we are introducing level db it's something like database it's very heavy it's a key value store we don't need even key value store we just need flat file key value store adjacent into the flat file it's done all of them in the java script so that's the way so we don't need any kind of database because it's very heavy i'm just thinking of 2 different methods one from flat files to your store and then from your store to your office it's 2 different methods i'm just telling the complete solution emat tell me now we don't talk about database we talk about operational without any kind of problem that's a different story i will tell you the complete solution so we get some custom in's to move the flat file we have some custom code we call it agent the agent is in the terminal also the agent in the store this agent maintaining the herbes maintaining the file transfer maintaining the synchronization also maintaining the updates we can update the applications in the terminal because we use git if there's updates from the developer we send to the what is it repo and this repo will distribute to all of the terminal and right away when there's changes the agent get the changes if the cashier press the print after print it's restart and restart new facility it's very smooth for fastening update we use git also the agent is customized it's a C application the agent is somebody who take care to glue all together we use C because it's very small server but cannot use any other than that ok, maybe you want to ask anybody? thank you very much for the questions if you have any kind of problem and wanted to discuss more fooder, you can reach me by mail me my email is can you see write my email i have folder to write so it is not connecting sorry my email is julianto at equinix.id just make sure that when you upload your slides you upload or send your final slides to me send them with your email id and when i upload them on slides here you all will have access to these slides PG Day Asia is an official handle for Postgres community in Asia to post all our slides over there so the contents we had yesterday and our first Asia track today all the presentations are going to be uploaded to that slideshare account you will have access to that do follow us on twitter and facebook as well we will keep posting about all these slides yourselves sleeping and awake on our twitter handle and facebook page now is the time to go downstairs for getting better pictures clicked post Asia is connecting a photo session so you will be part of a very large group so come downstairs can i have a photo season first i want to copycat the dream okay i