 Now we start the second session, that is we run a practical one-in-a-rization with AIDA. So after this tutorial, I hope you are much more familiar with AIDA. So I will start with an introduction of using AIDA for calculating quantum dispersal band structure. And in this tutorial, first tutorial, we will have a look at these basic AIDA data types, and also these workflows, and the job submission, job status, and the result analysis using AIDA. And then we will launch a one-year-nightly bands working to use this ACDM method to calculate one-year interpolated bands. And finally, I will give a brief introduction of our latest work that is to use a different automated initial projection, this entanglement method for high throughput one-yearization. So let's first have a look at this submission script for the PW band structure. As you can see from the left figure, it's only 24 lines. So with these 24 lines, you can already get a quite decent quantum dispersal band structure at the right. So this is the power of AIDA. So we will go through line by line of these 24 lines and help you understand the different aspects of AIDA. So first, let me remind you that we have three different interfaces to interact with AIDA. The first is this Verdi shell. It's actually a customized high-pattern shell. So we can use this Verdi shell command to initiate this interactive high-pattern shell. And then you can, for example, import these AIDA modules. The second one is this Verdi command line interface. So we can use this Verdi command line, for example, use this Verdi status to have a look at your configuration of your AIDA. And also we can write a Python script to launch your AIDA workflow. So let's first start with the first part of the script. So essentially, we define a code that equals this QEPW6.8 at localhost. So what does this mean? So first, let's have a look at these basic AIDA types. So in AIDA, to automatically record the provenance, we have some basic types like this orn.int and orn.float. And also we have this orn.computer and the code to model physical computers and executables. We also have this band data and trajectory data to model these advanced data structures for band structure and also this relaxation trajectory. And also we have the types for calculations and workflows. So these types are like the Python class and the nodes in AIDA is like the object. So for each node, it has this pk that is the primary key, which is an integer to represent the node. And also it has this uuid, which is this universal unique identifier to help you identify your node. And also some nodes have this human readable label. Like for the code, it has this, for example, the code has this label that is this QEPW6.8. So we can just use this label to identify your node. So in Verdi Shell, we can use this orn.load code to load your code into this Python environment. Also we can use this Verdi command line interface, this Verdi code show to show this configuration of your code. Like this is the absolute pass of this executable. So the second part of this grid is to read an atomic structure and convert it into an AIDA structure data. So here we use the ASE.read that is this read function from the ASE package. Then we convert it into this AIDA structure data. So we have this automated provenance. Then we store this structure into the AIDA database. So the third part is we use this PWBansWordChain to get a builder from this PWBansWordChain. So essentially you pass the code, the structure, and we're using this fast protocol to get a builder. So what is this builder? So we let me remind you of this workflow in AIDA. So we use this cal function and the WorkChain workflow function to decorate this normal Python functions. So we have this automated provenance. So in this for a complex work chain we use the Python class to define this outline to run a practical workflow to calculate for example the band structure. So the WorkChain builder is like that. So the AIDA will automatically populate the input parameters according to your code and your structure. So if you want to change the predefined input parameters you can just modify the builder. The builder is like a Python dictionary. So you can change everything before submission. So for example in our code we skip the relaxation by remove the relaxed key. So we can have a closer look at what is the builder. You can print the builder is essentially like a dictionary. So it has this bands key which contains the as you can see this PW.x input parameters and also contains the number of machines that will be used for the MPI run and also this max work log second also contains the suit of potential. So the builder also have this relaxed part. So if we have this relaxed and defined simple parameters that AIDA workflow will run the relaxation but for because we want to calculate the band structure so we just remove this relaxed part. So AIDA will skip this relaxation. Also it has this self-consistent part but also it defines these input parameters for the PW.x. And so to that we can change this builder before submitting this to the AIDA daemon. So the last part is to submit the builder. So after you run this submit builder the AIDA daemon will take care of creating the input file the real input files and then upload this files to the local or remote cluster then submit to the slurm pbs or these job controllers and also monitoring the job status. So once the job is finished it will retrieve the output files parse and store the results into the AIDA database. So when your job is running you can use this very processed list to list all your jobs. For example this is the job this one year 90 bands working that I launched 18 days ago. And also you can use this very process report to show the each step inside your working. For example the first step is the self-consistent step then it launched this bands calculation. So finally everything finished successfully. So we have band structure to visualize the band structure you can use this AIDA function this very command line interface this very data bands export. So you have this a pdf file for your company's special band structure. So as you can see with these 24 lines you can get everything this band structure from this a single crystal structure input. If you change this bounding to any other structure you can work well and give you a band structure. So in my opinion this is very clean for anyone even if you don't know quite a lot for the the input parameters of the quantum dispersal we can still get a decent band structure. So in our tutorial we will launch this quantum dispersal band structure and then we will also launch a one year 90 band structure. So the one year 90 workflow implement this SEDM algorithm so the SEDM is implemented in the pw21990 codes and this we use the projectability to determine this and mu and sigma for the SEDM parameters. The workflow is implemented within AIDA and also in the hilarious paper we validated 200 crystalline materials and also 81 insulators to evaluate the interpolation quality and the result seems quite good. So on materials cloud you can find these all the problems the full problems of all our calculations in this automated high-through group minimization entry. So everything is on materials cloud. So for example again find select the structure and then have a look at this crystal structure and also these calculation results the band distance of various methods. Also we can compare the DFT band structure with the one year interpolated band structure for isolated case and also for entangled case. So thanks to the full problem and so we can track everything that we have done with this automated one-year realization. So for the one year 90 band structure work chain the band's work chain this is we specifically designed it to be almost the same as the pw band's work chain. So instead of a single code now we have this pw.x project with function pw21990 and when you're 90 but all you need is only to pass these codes into the work chain and we use the same interface so codes the first is code and then structure and the protocol we use this fast protocol to speed up the calculation. So it will use a set of reduced parameter to reduce the computational time but as you can see this is the same as the pw band's work chain. So in this second tutorial you will launch this pw1990 band's work chain to compare your one-year interpolated bands with the quantum dispersal bands. So finally I'd like to talk a little bit about our recent research on automated projection disentanglement. So different from the ICDM one-year function one-year realization. So we develop a new algorithm to do the automated one-year realization. So if we think about how do we choose the ICDM parameter so we use this projectability from the project with function.x so essentially we project the block wave function psi onto a set of localized atomic orbitals inside the pseudo potential this GN. So this GN inside the pseudo potential so if we look at the initial projection of the one-year 90 it also may use this local set of localized atomic orbitals but here we're using some analytic orbitals from the hydrogen the solution of the hydrogen Schrodinger equation so why don't we use these numerical atomic orbitals inside the pseudo potential to use this as the initial projection for one-year function. So these pseudo potential numerical atomic orbitals are for the wavelength electrons they contain the angular momentum information such as this spd and f also it contains the radial part of the wave function for each element. So in principle it should be better adapted to a different element than using a single hydrogenic solution. So after we exclude the semi-core states we can use these projectors as the initial projection for one-year functions. So in fact we can create a table of initial projectors for all the elements to do this automated one-year realization. So I'm not using these pseudo atomic orbitals this abbreviation as seen in the paper I'm using this NAL because apart from pseudo potential projectors we can use different type of numerical atomic orbitals for example here I tested some orbitals from this open mx package which is a linear combination of atomic orbital package so when usually when we when you write the silicon band structure we can only have a good description a few electron volts above the conduction band minimum. So if we include further these d orbitals into our one-year functions we can expand much more the description of conduction bands as you can see when even this graph accurately 30 electron volt about the conduction band minimum. So this tells us we can further explore different projectors to improve the flexibility of one-year initial projection. Also we can use the projectability information to do some different disentanglement so for example in this graphene case on this left panel this is the projectability the band structure projectability of the graphene. So as you can see there are some atomic orbital like bands inside these three electron bands these colored bands have high projectability on these atomic orbitals well these gray lines are like three electron bands so if we have a window disentanglement that is this this froze mech or this froze mean this wing max and this wing mean so we can only use a horizontal window to freeze the states instead we can use the projectability information to freeze only these atomic orbital states so on the right the right most panel you see if I use projectability to freeze the high projectability states we have you can recover much better these atomic orbital like states that means our one-year functions are more restoring the atomic orbital picture of these band models so here we have these two parameters the p-max which if the projectability is higher than this p-max we freeze the high projectability states well if this projectability is smaller than this p-meh we exclude this from the disentanglement process so in with this trick we can exclude these three electron bands and only do a one-year function from these atomic orbitals that these atomic orbitals are not universal across different chemical environments so for example for silicon the conduction band minimum has low projectability so it's like around 80 percent so it's not always if we only use projectability disentanglement it's not always you can find that here the conduction band minimum are not well described so instead we can use a combination of projectability disentanglement with window disentanglement so we freeze for me for me energy or conductor conduction band minimum one or two electron volt above this tool and then we use projectability disentanglement to freeze still high projectability states in this conduction band region so this will help us better restore these atomic orbitals so instead of tuning that this froze max we now we only need to tune this p-meh and p-max so usually we only need to set this this froze max as the cbm plus one or two so this so we further do some quantitative comparison of our new projection with the with respect to the icgm projection so to compare the interpolation quality we use two measures the first is this average band distance and then this also the maximum band distance this average band distance like the square difference and then this root so if we change this fictitious new for example we set this as fermion g plus one electron volt then we are comparing bands below fermion g plus one electron volt so so here we so as we we have seen this projectability this p-meh and p-max is kind of unspecified parameters so how do we choose this we further implement this optimization mode change in aida so we can so we can find the optimal p-meh and p-max by sweeping all these parameters for example here we have this optimal band interpolation error and this combination of p-meh and p-max or for other material it might be quite stable so you can either choose one of these p-meh and p-max so with this new projection we compare the band distance between the icgm linearization and also the new method as you can see the band distance is more concentrated towards zero so quantitatively you can find the band interpolation error is better than the icgm and also for these eta-max from 300 to 60 mean electron volt and moreover when we compare the spread distribution the new projection almost all the linear functions have spread less than four angstrom squared so while you run this icgm linear function it has some large spread linear functions so with this new projection we are we are happy so we launch around 20 000 structures to do a real high throughput linearization so as you can see from the result this so now it's over 17 000 so it's like we have very accurate band interpolation really on the linear from volt also so finally i didn't show other statistics so so finally today we're not going to use this new method because it's just still not fully incorporated into the quantum espresso so today we're only we'll use this icgm linear function linearization to launch a automated linearization so you can use this command to copy the tutorial's files into your folder and activate your virtual environment by this work on aida and then i prepare two script this the first is to launch one year py so you can pass it crystal structure and launch your linearization also there's a launch pw band so you can calculate your pw band structure and also i prepared a tutorial or we prepared a tutorial on this this web page so you can follow this tutorial step by step so it has a very detailed explanation of all the steps to do a automated linearization so you can follow all the steps and finally you will have a band comparison like this so you can really see the accuracy of the automated linearization and and finally if you have any questions or issues in the future just need to create full request or create issues on this aida repository and also you can ask questions on this aida mailing list so we will help to solve your problem and that's the today's tutorial so i think you can start working on this so everything should be ready there okay now we just need to copy this two files copy these folders and also the link for the tutorials in this readme so you can click this link and start your exercise also there are some optional questions in this web page so if you have finished your tutorial you can try to answer these optional questions yeah so that's all for today's tutorial thanks thanks okay so there were a few technical questions that were asked during the presentation of your friend by Giovanni Pizzi on the chat Giovanni maybe you can summarize very briefly the questions you were asked and the answer you gave sure so let me just go back I think the first question was very practical which was what was the meaning of ORM the acronym the acronym stands for object relational mapping object means a python class and relational means a database table because on the on the back end either we store everything out of the database in order to make it possible to query easily the results but what do you see in python are python classes and so there is a mapping that says okay a data node becomes this in a table and so on and that's why it's called an ORM it's a very standard name and computer software engineering let's say in the databases the second question which is a bit more general actually a question we get relatively often is how can if and how can i either connect to supercomputers which requires some type of a multi-factor authentication or two-factor authentication something like you have to press something on your mobile phone or you have to put OTP one-time password code like on Google Authenticator or you have to put some numbers in general my answer to this is as long as it's possible to do it automatically somehow 99% of the cases it will be possible to do it with either if it's not possible technically because it's disallowed by the supercomputer to connect via without a human doing something or because by policy it's not allowed maybe technically you can still do it but it's not allowed by security policies then it's not a problem either it's a problem of any script even if you write a bash script that wants to submit things you would have the same problem so the first thing would be please check with your supercomputer center if and how they allow script not a human to connect to run simulations if it's about a one-time password it's possible there are online on github there's a script which allows you to return automatically the six digits you have to put and this can easily be integrated within aida if it's something else again it depends practically very often the solution to this if the security people don't allow it is to ask them to have a computer a node something you can access to where you can install aida inside the supercomputer network in this way you have to use the otp by hand when you want to connect to aida to check what's going on but then the machine when either where aida is running can connect to the supercomputer directly without otp because it's already inside the network so you shift the program but then you solve this issue and then aida can connect automatically and submit jobs um i think this should address both questions there's a very technical question now just asked by alba to which junefang just replied if you don't find the commands like run aida or the verdi command line remember you have to run this what you see here on line three on junefang slides which is the command work on aida which enters what's called a python virtual environment is a virtual environment like the name says where you see all the python packages about aida and the dependencies this is just to isolate all the dependencies from other codes you've been seeing you just run that then you see something on your command line you see something tells you are inside this aida virtual environment and then from there on you see all the scripts the commands okay i think this this was uh uh more or less what was asked on zoom are there any questions we are interested from the present participant raise your hand anyone okay so if not what i would do is to stop the recording now for the lecture