 I'm Jörg Steppens from the Barriers Projects and I'd like to show you some methods to interact with barriers. I assume this talk makes most sense for people who are already using barriers or a regular. I hope you're all using it. We will see. Then I will start with a short overview about barriers and then I show different interaction methods. One way to interact with the system is to configure it. I will handle this shortly. The other thing I guess you are more interested in is runtime control of barriers and the other way around things that get triggered by barriers afterwards. How does a barrier infrastructure look like? You have a central director located which keeps all the data. It controls all your backup environment and stores its information in a database. You have the storage daemons which you have one or more multiple storage daemons each with one or multiple storage devices attached. This is where your backup data is getting stored. You have your file daemons or your lines installed on the different machines that you want to backup. You can interact with the director during runtime through the barriers-director-console interface. This is used by command line tool or web interface or some web tools. This is the same just with newer lines. There can also be some plugins like VMware, MSSQL, NNP, cluster, etc. Compuration file. Compuration is handled in resource. The director configuration consists of a director resource, a catalog resource, how to connect to the database. Your lines, the file sets, so the data gets back up, your drops, who is scheduled to storage. As an example, I put in here the drop resource. Each resource got a name. In this case, backup Barrios FD from type backup, level incremental, and then it references to other resources. So it references to the client Barrios FD resource. It refers to the file sets, Linux all resource, to the schedule, and so on to this different pool. And it is enabled, so you can also disable it. I missed some lines here. If you're not sure what you can type in here and are not willing to read the documentation, you can always ask the demons itself. So, for example, the director Barrios director minus XS, like export schema, will show you all available derivatives as you can configure. If you change a configuration on the director, you can reload the director. So it's then immediately working for storage demon and pile demon. You have to restart the demons. I hope this has been everything on this slide, but I guess so. Confuration can be quite large, but it's relatively straightforward, and there are modules available for individual chef puppets, so I'm not aware of. So far for the documentation. Confuration, sorry. So runtime control of Barrios. You can connect to your director. He has a D-console command that you would like to handle this on the command line. Type in D-console. It opens up a TCP connection to your Barrios director. Authenticates after this gets you an interactive prompt that you can type in your commands. The set, you get a list of all available commands. But you're not limited to interactive reuse. You can also pipe in some commands in, and then they will be executed, and so your designs will be listed here to standard out. So there are quite a number of scripts that just use this simple interface. There are some helper functions inside of D-console to generate, to write that script. Like, for example, these are the so-called add commands. So you can redirect the output directly to another file, and also handle this part of, this command should be outputted to this file, and if you have another output, then the rest will be outputted to another file. At this point, you can do the same as input for commands, or some other commands are available. Most commands have also the parameter yes, which says, don't ask me any questions, just do it. So confirm, immediately confirm, and if you have missed some parameter, then the command will not start. Rate is also important, rate until the job is finished, or in this case, until all jobs are finished. As said, you have, this uses the direct-console interface, and this can be accessed by D-console, which we have seen right now. There's also a Python module, Python Barrios to access this. So the Barrios web UI is the screenshot of it here. This uses the same connection, this is written in PHP, so if you have programmed something in PHP, you can reuse part of the code, and Q3-based interface we, but also uses the same interface. But this is much as deprecated across the whole thing you need. This is implemented in Barrios web UI. You can have it wherever you like, because you just use the control interface, and you can connect to it. In all ways, you will connect to it via TCP, so it can be on any system. Yeah, sure. Yes. Sure, but therefore I have to say, mostly it's easier to SSH to the machine, and then use the decontrol part, and also use it remotely. This is, for example, done on some clients. If you want to give them specific access to trigger its own job or something like this, then you can have it decontrol locally installed, limit the access rights, and use it from the client. Also, cataloged database can also be on other systems. Okay, I've talked about the control interface. You should be aware that there are different type of controls, so called default console or root console, which gives you access to all the sources available in the director. This is normally the console used when you just type in decontrol. So the default configuration, this is the absolute console. The other console is the name console or restrict console, and there, to use this, you have to define ACS for this console. So in this case, we have a console, and we're able to name user one, provide the password, in this case, secrets, and allow the user one to execute all commands except of the need commands. And only access data from the My Catalog database, which is the default database, so it's probably all, but you can limit it. And only to the clients, client one and client two, and all these words are regular expressions. So here in this case, Fileset, all Filesets starting with the name Linux and then string job IDs also. If you don't specify SEL for some resource, you don't have access to it. So you have to specify it and provide all or some things that you want to assign to it. I want to do some of the important commands of various help. It's always important to get the list of all commands available. If you also want to know the parameters, you type in height and then type help and then the command name with these 30 parameters. For one-time information of your different components, you can ask status, status director, so your information from the director, or you can ask status of one of your storage systems that's attached to it. Or if this is a tape library, then you can ask what volumes are assigned in what slots or ask the status of different clients. You can also ask when this scheduler has planned to execute this specific drops. If you don't specify a drop name here, then you will see get a list of all drops. Or otherwise, you can see one banner drop on a specific drop. This plan to get started. The list command is to retrieve information from the catalog, from the data base. You have two versions, the normal list and the long list, which provides you more information. And you can ask about information about your backups, your clients, height sets, and so on. For example, if you're interested what drops are stored on volume full one, then you just ask, list the drops that are stored on volume full one. Or the other way around, if you're interested, where the data from your drop one to three is stored on what media and you ask for the drop media from this drop ID. Or if you want to get the information you normally see at messages, you can also ask for those messages that have been created during the drop, as the drop did run. This drops last is quite important, I think. If you want to get a quick overview about the status of your drops, then you can type this command and it will show you the last one of all your drops. So normally, if you don't do this, then you maybe just get all drops from the last 24 hours or something like this. But with this, you also get dropped that didn't run even for a longer period of time. But sometimes confuses people on the mailing list is, for example, if you modify or add a file set and then restart your director and then make lists file set, then you will not see it. Because these data is first synchronized to the database and there has been a drop one that uses this file set. Same with slides. This slide will only show all the slides. So we have a drop did run for this slide. So if you have changed your configuration and want to verify that the director is aware of this, you can use the show command which shows your configuration. You can also ask for specific resources in here. Yes. Yeah, and then you have commands to execute jobs like backup jobs, restore jobs or wait for a job to be finished. A small example to put this together is let's say you have already installed your client's file demon on a new system and want to configure it on the director. You can use the configure ads clients, give it a name, give it a address and give it a password. This command will complain if you have missed some required parameters or if you have put in invalid data. Otherwise, it creates a resource file, a configuration file for the new client and immediately loads it. So you can directly start to use it and check if it's really working. This starts as client and then client name. The same you can do with jobs. So you create your jobs and after these two commands, three commands, you can respond to the job. No, not here. Not if you use this command, then the director creates it automatically. You can also specify all these specific parameters, but if it's a common approach of them, you only have to expand by hand. Yes. It will create a configuration file for you and immediately loads it. That's one of the advantages. In this case, it also creates part of the configuration file that needs on the client. So if you look the other way, once you first add it here, then it has also director export directory and you can copy it over to the client and you start it. No, not with this because you have not the knowledge about this. You want to add some scripts that had to use this, but they haven't done so. For Windows, we have the installer and this has silent install options that you can use to configure it. Also with the director, with the password and things like this, you also provide Opsi packages for the Opsi because Windows management, software management system Opsi, so you can integrate it in your Opsi environment, which you hopefully have and then install it and configure it. This is all things that work well, sometimes they don't, so you can also enable debug during runtime. You do this with set debug, specify the level, 100 is a good starting volume. You specify that it should log to a log file and adding temp, temp is also usable and then what components should have the debug enabled, in this case the director, you choose the storage, what this is. You can't specify to what file, in what file the log file will be generated. However, if you discommand, it will write you the file where it's locked to. This is not a big deal on Linux because it offers in this directory, but on Windows you are glad because depending on the Windows version, to this difference log files. This has been now all about the normal commands. You have also the special dot commands and they have been intended for non-interactive use, but if you're waiting for it, then you might want to use them. You will not see them with the help command, but if you use the dot help command, you will also see these dot commands. For example, dot SQL, you should be able to get all the information you require by the list command, but if this is not the case, you can also specify arbitrary SQL commands to execute. Of course, you could also directly go connect to the database, but here you have already the authentication to the database and don't know what type of database you're using. Now it's getting interesting. The normal output from the v-control is in rp-mode 0, meaning rp-mode disabled. So it's intended to be human readable and it's quite okay. I guess for that some time ago, p-mode 1 has been introduced. There's a reason why we haven't used this anymore, but we introduced ap-mode 2 with output data in JSON formats to be better usable by other programs. So as an example, let me start on top. In ap-mode 0, list-drop 1, you're getting this table nicely formatted and gets the information in there. This ap-mode 1, the declaration is stripped, the first part is stripped and I'm not sure what else has changed, but unfortunately this is not really consistent between the different commands. This ap-mode JSON is the same command. You're getting a dictionary named result and insights because we asked for jobs. You're getting an array of jobs. This is the information as the fields are called. We already tried to mimic JSON ap-c 2.0 format because we have to use some format and maybe then it gets easier to extend it someday to a JSON ap-c server. Yeah, how to use this? For this you could use Python variables. Python variables is something like the deconsole, so it first connects to the director, but there's also a sub-module for JSON, so you can handle the JSON data immediately. You can also directly connect to the storage daemon or file daemon, but they don't have... Yeah, they only provide limited commands that you can use. So as the same example as before, you first have to prepare your password, you import the various BSOB class, prepare your password, then use the director-console-json class, connect to this address with these users and this password, and then you call the command that they are issued before and the humidity gets returned a Python structure with the data encoded. Also not the result text, just a re-result. If there is an error, then you get an exception, so this is quite usable to create your own scripts. If you're doing backups, you probably care about the files and I will now show some methods to access data from files. So even before you really did run a backup job, you can call estimate, listing, and get in return, so then the director will connect to the client and this will return all the files that would get back up if this job would run. If you have really run this job, then you can, with list files of this job ID, you get the list of the files and directory that are back up by this job, but without extra information, so start information and so on. If you use the restore command, say the last backup from this system, then it will generate a virtual file tree and will give you a new prompt and then you can use dir or ls or cd command to walk through your backup data. If the directory is not really included in the backup, but only this because it's the parent directory, then the values here are all zero, like in this case, but here you also got the real backup data from your files, you can mark the files, say done, and then these will be restored. This is fine, but if you really want to write the script to interact with your file data, then this is not enough. For this, you have to use the BVSS API. This is described in the developer guide and it looks quite nice, as you see. You first have to update the cache for the drop ID you're careful and then ask where does this backup start. So BVSS Ls directory is from this drop ID with an empty pass. You'll get root directory in return. This root directory has pass ID three and other values are set to zero. And AAAA means this is all also zero because what you see here is serialized form of the start field normally. This is done in this way, and stored in this way to be more flexible about the database, about different lines, but it's not really handleable. But okay, you now know that your slash directory is at pass ID three. You'll get the directories of pass ID three. Then you'll see, oh, there's a user directory still with no information, but it's got the pass ID two. And then you look for the directories in pass ID two. Then you'll see, oh, this is really something because here are some information and the user sbin directory is in pass ID one and there are no further sub directories, but you can list the files that are in this directory. So you need BVSS if you want to ask various directories for specific directories. Yeah, I say that's not really easy, but this is the way that and also the web UI restore browser did work and use the sub commands because they can't work with the list of all files because they can be quite, quite a many. Oh yeah, in JSON format, you can at least read the data. Yeah, if you want to, but it doesn't help too much. Another way to access this is using Barrios Fuse. So a Fuse file system based on Python Barrios, JSON mode, and if you mount this, this will connect to your director. And in this directory, you mount it to Barrios FS. For example, you get directories for clients, for jobs, for pools, for volumes, and then you can list, for example, the jobs running directory and you will see, okay, currently there's one job running, has this job ID, has the name Gonzo, client Gonzo FD, level is full on status is running because otherwise it wouldn't be in this directory. You can not only ask for running jobs, but you can also ask for all the jobs for client Ting FD and then you will see, let's call it to here, you have on the 20th January, you have a job with this job ID for the name Ting, client Ting FD, level full status, terminated successfully, so this is good. And the day before, you have a similar job, which is incremental. I guess they both started at the same time, you see the full job ends quite later, two hours later, and so on, and here on the 70th January, you get a job that it failed. You not only see the job, you can also go to the directory and then get access to the job block, for example, or the info, which is the information you would get from this job ID of this job and the data, you can descend to the data directory and then you will see all the metadata or all the files that you have back up in your backup job. Of course, you can't access them there because you only, you haven't done a restore, you just look at the metadata that is stored at the directory in the database, but you get the information, you get the metadata, the file change date and so on, and you can use your normal UNIX tools like finds, for example, to find all the files that are larger than 100 megabytes. Incremental will only show only this specific job. You can handle this otherwise, this is just really shallow Python code with list jobs and list files and so on. If you want to have other features just, it will be few lines of code, three or four or something like this. You not only can see the jobs of specific clients, you can also get a list of your pools and your volumes in your pools, then you can immediately see we have this always incremental full volume that is in status full, so we have written this amount of bytes to this volume and it's in read-only mode because, yeah, because it's full. And here on this other volume, you see it's an append mode, so you can still write data to it and therefore it's in read-writes. It's last time data is written to it, it has been written to January. Again, you can change to the directory of a volume, you get some extra information and also drops so you can descend to a volume, to a specific volume and then the jobs and you will see all the jobs that are back up on this volume and in this job you can change to the directories in there and get the files that are back up with this job. So quite useful and you can write your normal Unix tools to script to retrieve some information. It's not the most efficient way to get the data but, yeah, it's quite comfortable. Okay, now we want to care about specific problems and writing script for this. For example, you have your various backup environment and you backup all your servers and everything works fine but your servers are always there so you can have a fixed scheduling for them and backup the data. But now I think you have your coworkers with their laptops and they are sometimes in the office, sometimes not and maybe you have dynamic APIs and so it's complicated to get time when you can backup them. So how to solve this? With this short script I can say and it works like it gets a list of all no, I have to start otherwise. We use the client in a connection so in this case not the director connects to the client but the client is configured to direct connects to the director and the director is then aware of all the clients that did the connection to it and have the connection and if there's a job for the client it will reuse this connection. You get the information with status director and look at the field connected client connection and you only care about the name of the client. Then you loop through all the connected clients. You have the trigger function you can call them by a call for example every hour or every minute as you like. You get all the connected clients you loop through the client so guess a job name associated with the client so if your client is called client one you look for a job backup of client one and if this job exists you check if there is a job for this client in the last 24 hours so if there is no job for this client then you just execute one the job name that you have there yes to confirm it and in result you get a job ID from the newly configured job and you can just print it out so with this short script you can solve the problem with your mobile clients that are sometimes at the office sometimes not just by running this periodically checking the clients that are connected and starting a job if no job exists for the last 24 hours so I have how much time left 50 minutes oh 20 minutes okay then I go no I don't go through this because it's not all on the slide if you have a question about pooling volumes and how to automate this in specific case ask me after this talk oh is that the question? yes no no this is for the time that the clients are back in your office connect to the director which is also located in the office and then you know that your clients are there and then you trigger the backup okay now the other way around things getting triggered by various for this you got one script you got quite flexible one script either you can call with a job a command on the console itself so you see various direct barriers commands you have options like they should only run if the job has succeeded or only if the job has failed and you can specify when they should run before a job after the job I guess one of the set on the one on failure only works with after the job or on windows after VSS something like this you can also execute arbitrary system commands with a command one script and then you have additional options if this command should be executed on the director or on that line where the backup runs and if the job should fail if the system command fails some examples for example if you want to create a virtual pool backup from your existing backups and store it somewhere else to an archive pool but don't want that this newly generated virtual pool is again used as a base for the next virtual pool then what you can do is create your virtual pool store it in the archive pool but after you have done so update the job ID because you know the current job ID where it is variable and change the type from backup to archive similar to backup drops they are not used internally for any other actions so you can just run this this is a console command you can also type it in the B console but you can integrate it with your job and it should run on the server and it should not run if this job before has failed no, not the same no what other drops are available are admin drops normally you get backup drops or restore drops and this admin drop you don't do any actual backup you just use this to run some commands and in this case after the backup of your own system you optimize the database which can be an advantage if you are using Postgres so do onto backups but just use this as a kind of con but we got my other things relax and recover some of you have heard a few are using this or have heard the talks before relax and recover is a disaster recovery environment for linux which works quite well it works in two steps once it creates a rescue image from your current running system it creates a rescue image and you can boot with this rescue image for the system second step itself could do backup but it can also integrate other backup solutions and thankfully varios is also part of this and a minimized way and quite comfortable way to integrate it with varios is normally you got the question ok, when my system broke I need my rescue system so I should be able to have a recent rescue system and then to create it and where to store it because normally you don't want to store it on your local system we have options to store it on NFS and things like this but we are a backup solution so this approach just creates this regularly on every full backup and stores them locally on our own file system because our full file system is back up to the backup server and if you really require it you can extract it from our normal backup and then we store it to another machine where a CD burner or a USB drive is attached we can put it on there and then we have the rescue image we can place it in the same server with the replaced artist and restore our system we do so with the one before script so before we actually do the backup we run on the client disk command this is a shell script this has one parameter this is the backup level so full differential or incremental we only execute this on full because if you would do this in incremental then every incremental have an additional size of 50 or 60 megabytes which you don't want but on your full backup with whatever 2 gigabytes you use 50 megabytes more or less are not really important and it's important to create this rescue image regularly because it integrates your current kernel, your current network configuration your kernel, your current hardest layout so if you don't do this regularly you might run into problems because if your rescue system is one year old so you can only get back to the status from your one year old system so again, something is missing on the slides what I would show you if the scheme has been larger you then have the b-console you type in restore of let me think control-minus control-minus is also an option hey, thanks ok that's what you have to type restore from the current backup of Client, you care about, Client FD1 extracts the file valyp-rear output-rear-barrier.iso this is the default location valyp-rear and restored to some other system which has CD burner attached and use this special restore job just we don't make the restore in the valyp-rear output location, but in the temp-direct you have some other directory you want to use so with this you have your normal backup drops and yeah, and your rear directly integrated with this it requires you, right? no, it doesn't require it, it offers you to restore on NFS, but it also offers different methods like maybe also copying Zumba for sure and a lot of other possibilities I think, yeah, so that will just boot up the ISO and then when you have the recovery system up then you restore the file from yes, so the rear-barriers integration is that rear you can configure your rear barriers and then in the rescue image creates, uses the kernel your network setup, your disk layout, but also copies all the back-barriers tools you have installed on the system, on your rescue system so it boots up and has the same file deamon available that you have on your running system and with this it restores just your data from your normal backup and that will automatically be already into the recovery ISO just as this one line back up barriers, something like this and this is all they would be in the boot command line no, this is in the configuration, we are local.conf, but it's just one line in the configuration so it's really easy to handle and really powerful yes, for sure okay, then this is about scripts, now we let's say two slides about plugins barriers can be extended by plugins for the director, storage deamon or file deamon with this you give that to specific events you care about on the file deamon plugins, normally you use them to backup specific data like a database, virtual machines or something like this plugins on the storage deamon are normally made for rewrite data like on the fly compressed data that's coming in normally this is done on the client but if your client is not that powerful but your storage server is then you can also make the compression there or SCSI crypto to use crypto tapes like this to give some status information or for the director this is normally if you write a plugin there then you do this for status information and the shortest possible plugin is something like this you have to import the barriers class for the director and then you have to overwrite the init function then you register for the events you care about, in this case we only care for the event which opted end, then you have to overwrite the handle plugin event function when this is called it's called with a specific event we check for this event this case it should always be event drop end because we only register for this but we check nevertheless then we retrieve some other information like what client the dropped it run, have there been errors how many bytes, how many files, how many bytes have been transferred we format that in some other way and then we send it to a singer or a graphite or whatever you use for monitoring or for visualizing your backup system or your environment so I'm mostly done just roadmap for the upcoming release for the things that's important for integration is the Python Barrios is currently available at the Barrios concept repository but will be moved to Barrios core repository in the next major release the Barrios web UI is actively developed by Frank over there and it will continue to be enhanced and therefore we also will extend to the vector for to provide more information yeah this part about media handling and optimizing this for this storage I skipped so I don't have, it's not so important what I missed here it's a roadmap we have seen here the console connection where you specify user and the password we have received pull requests for adding PEMP support which has quite some implication but we want to put work on this and get this integrated for we plan to integrate this also for 17.2 so you can directly authenticate your adapt directly or whatever you like okay now we got time for questions no questions I'm done good oh okay one question so for the Python Barrios module which opens a TSP connection to the director this also works with Python 3 and is already uses Python 3 for the plugin yes as I heard before there is only Python 2 right now but it's already prepared for Python 3 and will also be done soon I guess yeah I guess per distribution because normally we compile it against what's available on the distribution and if the distribution only offers Python 3 we will do it for Python 3 okay other question yes in principle it's yeah it's connected to the director and then you can call the call and put in all the commands you could also enter the B console and so this will be executed and the results will be given to you okay hello how did you know how did you know how did you know how did you know how did you know how did you know how did you know fail over if you want to have a fail over the only thing you need you have redundant database that you do with your database tools and then you can have a second director for this is only important the configuration director ETC Barrios Barrios director and you can copy them over or keep them synchronized and then you can start the other director you never tried we tried it in one project or we did it in one project okay but now not simultaneously so it's just a fail over it is possible that you need to expose those to the internet because maybe you don't have everything in your lab if you are doing remote backups is there a security concern for you of course it's a security concern if you provide services to the internet but yes we are aware that people are doing so you of course should have your TLS encryption configured to do so and probably also what would be a good choice is also to encrypt the data on the client there's also a various option so the support is encrypted and also through the encrypted transport your data is also have been so this should be quite reasonable so all the communication is encrypted and the data the data is encrypted the data is encrypted on the client and can only be restart on the client or if you got the master key also configured somewhere else okay and the second question is the very bare minimum requirements on the client you know to keep it lightweight is having the file demon storage the file demon, only the file demon if you want to keep it as small as possible that's the only requirement you have on the client how resource hungry is that the demon storage on the client like is it very, like do you need a powerful machine, like do you need they think that the director needs to be four gigabytes I think was the best practices I'm not sure about this I know that some people wanting the director and also the storage team so I'm sure that's not so many of course it's not the fastest solution but a lot of people using this this way and also the core components are all written in C, C++ also there you are running on AX APOX from 10 to 15 years old so it's not so so intensive even with big storage like very, very heavy if the data set is growing so if you have to detect the files that have changed since the last backup then this can consume some time and also encrypting the data on the client will also consume time but this is normal so if you have to if you cannot afford this from your CPU time you can configure it to make this like maybe unencrypted but yeah maybe you don't want so okay this is it oh sorry they are planned so the question has been to use some file system management to detect changed files more quickly and this is for ZFS or BetaFS yes and we discussed this and we have also a task defined for it but currently nobody is working on this but yeah it could be hopefully done quite easily but I've checked this with BetaFS and this wasn't this easy because if you want to retrieve the file information that has changed these virtual mount points and then you yeah it wasn't that easy it's easier to get to change blocks but we're not backing up blocks, we're backing up files so also this is not too efficient but yeah with a file demo plugin this can be implemented quite easily okay