 OK. So good afternoon, everyone. This is a presentation on how we use OpenStack on the EGI federated cloud. I'm an orphan, and it's working for EGI Foundation as a cloud architect. And I will speak to you first about EGI, because probably most of you don't know about it. What is our federated cloud and also how OpenStack is used there? So first, EGI. EGI is providing advance competing for research. And what we try to deliver is two open solutions for science and research infrastructure by federating digital capabilities resources and expertise between communities and across the nation and other boundaries. So how we do this is by offering a set of services, which are listed here. We have three big categories. First one would be compute. And here we have cloud compute, which would be infrastructure as a service, so running VMs, block storage, and these kind of things. Then we have cloud container compute, which is running Docker on top of the cloud compute, actually, on top of virtual machines right now. And we have also high throughput compute, which is providing you an enormous patch system where you can execute thousands of computational jobs to analyze data sets. Then we have storage and data. Here we have online storage, where we would have the object storage, like Swift, and also a file space storage that is used for some of the research activities. We have archival storage. It's about making backups, basically. And then we have also that transfer service that allows us to move very large data sets from one place to another in a reliable way. And the last kind of services that we have is about training. We have two kind of services here. We have the FITASM training. FITASM is an IT service management standard, which is lightweight and is focused on federations. So we do training on this kind of service management standard. And we also have a subset of the compute and storage infrastructure for training. So if our research community wants to do some training event, we can let them use some of the resources for these kind of events. So they get new users and these kind of things. All of these services are provided by the EGI federation. So the EGI federation is composed by 26 partners. 24 of them are the National Grid Initiative, as we call it. So basically, institutions from each of the European countries. And then we have also CERN and EMBL as part of the EGI. All of that is managed centrally in the EGI foundation, which is located in Amsterdam. And this was established in 2010 after a decade of investment by national governments and the European Commission. And we cover the whole Europe. And we try to support cutting-edge research, innovation, and knowledge in Europe. Most of the resource providers that we have are from public institutions or research institutions at each country or universities. We have currently one public cloud provider, but mostly it's this research institution. And these are the centers that provide the resources for our users. The EGI federation is, I think, the largest distributed compute infrastructure worldwide. So we cover the whole Europe, but we collaborate with international partners. So we have right now more than 300 data centers. 23 of them are cloud providers. Last year, we had more than 250,000 VMs running. 1.7 million jobs each day, which have accounted for 2.6 billion CPU hours. That was a 26 increase in the last year. There are more than 48,000 users, also a 25% increase last year. And this has given results for publishing more than 2,000 research papers in the last year. So as I said, it's not just Europe. Although most of our resources are in Europe. We have partnerships with international institutions. So we have Compute Canada. We have the Open Science Grid in the USA. We have Africa and Arabia. We have Latin America, China, India, the Pacific region. We have also Ukraine and EGI in the middle in Europe. And what we do is we sign the memorandum of understanding with these partners to deliver our service in a coordinated way. Our users come from a wide variety of areas and sizes, I could say. So we have from the very big groups of the very big scientific collaborations like the Large Hadron Collider at CERN and several others. Then we have communities that are not so big but cover several countries. So that would be the second column. Then we have also industry and SMEs involved in using our resources. And we also try to cover what we call the long tail, which would be the individual researcher or small researcher groups that also have computing needs. And we try to serve them equally. So this is all about EGI. Now I want to go into the EGI Federated Cloud. So the EGI Federated Cloud is a collaboration of communities developing, innovating, operating, and using cloud, federations for research and education. It's a small subset of all of our resource centers. I mean, it's 23 providers from 14 different countries. Most of them are open stack right now. It's 16 of them. Six Open Nebula and one in Cinefo. Cinefo is a Greek technology similar to open stack in Open Nebula. And right now we have around 7,000 CPU cores. So not big but hopefully growing soon. So what we do in this cloud federation, so we have this set of resources distributed all across Europe or across the world. And what we do is, first, we provide a harmonized operation of their resources. So we federate by harmonizing the operations. What that means is we have a single service registry. We have an information system where you can get information about the resources available. We have a virtual machine marketplace. We are able to collect accounting information of what's been used. And we have also a harmonized access control. So I will go into details of each of them in the next slides. And for the users, what we do is to provide uniform user interfaces. So we have the concept of Reum. And in what Reum, we say every provider of a given real mass use the same interface for the users. So we have two. Right now we have the open stack which uses open stack as an API. And we have the open standards Reum which uses OCCI, which is an OGF standard for providing access to the users. Open stack is supported just by the open stack sites, obviously. And the open standards Reum is supported by all of the other kind of technologies that we are supporting. So now I will go into each of these federator services, this harmonized operation, and how we integrate that into open stack. So first I will go into the EGIAAI, so the authorization and authentication infrastructures. So right now we have users identified with X5 format certificates from the IGTF Federation. I don't know if you are familiar with that. But OK, users have certificates that are extended with what we call BOMS extensions. So BOMS is a group management system, basically, that provides attributes on membership of a community. So if I am member of a group organization, this software will say, OK, you are a member and you have a given role and a given group in that organization. Problem with this is not very user-friendly. People are normally complaining about certificates. They are not easy to obtain. Then it's not easy to use them in web-based graphical user interface. So EGI is moving towards other ways of authenticate users. And we are moving to what we call EGI check-in, where we are going to use federated identity standards like SAML or OpenID Connect, and ideas to allow users to authenticate with their institutional accounts. So if you have an account at your university, you could just use that to access EGI services. And we plan to go BGN BOMS and also integrate other attribute authorities if a community has such. So how we do this with OpenStack? For the certificates and BOMS, we have a plugin that we call Keystone BOMS. This is a whiskey filter for the Keystone version 2 API. So you deploy this filter in your Keystone, and it's able to extract information from the proxies coming from the user to get information about the user name and the groups that this user belongs to. And automatically, it will manage those users for you. It will create the users in Keystone, and it will add rows to those users. And these mappings of the BOMS attributes to Keystone is defined on a file. This is available on GitHub, and anyone can use it. So this is working right now. It's fine, but it's only for version 2 of the API, and as I said before, it uses the certificates. So we are moving into a new way of handling this, which is with the EGI check-in. So in this case, the idea is, in this case, is an example with Horizon and SAML. So the user would get to Horizon. Horizon would contact Schibelet in the Apache HTTPD that's running in Keystone. That would redirect the user to the EGI check-in. And in turn, the EGI check-in will redirect the user to his home IDP. So this could be wherever the IGTF for if the user has a certificate due to J2Gain or any other IDP. The user entered the credentials there in the IDP, and the IDP will return back a SAML assertion. With this, the EGI check-in can contact the attribute authority and get additional information about the user, so which group the user belongs to. So the EGI check-in will extend the SAML assertion with create a new SAML assertion with new claims, saying this user is whoever, and it also belongs to Group A, B, and C. And this will turn back into the Apache to Keystone that will generate a token that can be used then to access Horizon. So we have this working. We basically are missing integration with some IDPs and attribute authority. We have all the technical parts ready. Right now, we are delivering to the resource provider at least this information, so the single unique identifier for every user, first name, last name, email, affiliation. And then we have the assumption attributes, the claims about the group membership that will be used for authorization in the Keystone. So this is ready, and we hope to get this sooner in production. Another thing that we have is a service registry. So all the centers in EGI are registered here in this central service registry, which is available in this URL, the .egi.u. And here we have basically static information about service endpoints, just to know who is providing services. We have two OpenStack-specific service types. We are having Nova and Swift there. So here in the screenshot, you can see the list, a partial part of the list of Nova endpoints in the EGI federation. And you have this kind of web front end and also API access so you can automate from your API query which services you could use. We also have an information discovery system. So you can get real-time information about the resources. This is based on a technology called the BDII, which is based on an LDAP. It uses a standard called the glue schema from the OGF also. And here we have actual capabilities, for example, which images or which flavors are available at the site, which groups are supported or the size of the resources. We have code for this also. It's available in GitHub. And it basically connects to your OpenStack-Nova API and gets information and publish it back in the correct format. Another thing that we use for harmonized operation is the accounting. So we have the central accounting repository. There we collect and aggregate usage information. And we have a portal where you can see this information across the whole federation. So we have also used a standard here, which is the OGF usage record that we have lately extended for cloud resources. And we have a software called CASO to get the information from Nova and also a salometer if you have it. And here in the slide, you can see this is just a screenshot from the accounting portal. The total number of VMs run in the last couple of months and ordered by the country and also by the community. So you can get this kind of information easily and publicly available. We also do monitoring. So this monitoring is basically health checking of the services that they are available and running fine. We do very, very basic functionality tests. So I am able to connect to the API. I'm able to spin up a VM and stop it. And it uses our service registry, GokTV, to discover what's available and do the automatic testing of what's published there. And with this monitoring, we are able to calculate availability and reliability metrics that are checked against the SLAs that we sign with our users. So if the users have signed an SLA of 95% availability, we're able to check with the system if the resource centers are meeting these requirements or not. And this is powered by a tool called EGI Argo. This is also open source available for anyone to use. And this is a screenshot of some resources. And well, you can see everything is green. You have 100% availability. If you have some orange or red, then some problems are there. So we have a very good view of what's happening and can react in case of problems. One, I believe this is a very nice things of the EGI federated cloud. This is the virtual machine image marketplace, which is handled by the app DB, the application database, which is a web application that has a lot of functionality. But one part of it is the cloud marketplace, where we have an open library of virtual appliances. So anyone with an account can publish images there that you can use on the clouds or download it and run it in your virtual box or similar. So you can reuse, share, as I say, the contextualization to this image. And I mean, it's good, so researchers can share their software easily. Here we have also a set of EGI endorsed images that are configured in a secure way. So we try to avoid problems in the infrastructure with these images. And they are tested, so we know that they will run everywhere. And we have also sets of images that are created by the community, by what we call the virtual organization. So a virtual organization can say, I like to have this image and this other image available for me at the resource providers. And when they select them, it will be automatically distributed to all the providers supporting the organization. Again, we have code available for doing that in GitHub. And here on the screenshot, you can see, well, this is a sample entry of a virtual machine image. This is EGI endorsed Ubuntu 14.04. So it's a vanilla Ubuntu 14.04, which is known to work in all of the resources and with everything closed, so you don't have clear security vulnerabilities there. This is the community view of the FDB. So this is a community called BioMet. And there you see some of the images that they have selected in their set. So all of these images will be available at the sites if the researcher needed. And the researcher can click on each of the images, and they will get some information like this. For example, this is one image that is available on a site called CessNet MetaCloud. And here you can see different flavors that you can use to start this VM. So from this single web frontend, you can get information about which images are available, where, and which flavors I can use for starting the images. And if you click on this Get IDs, you will get all the IDs that you need to just go to do your command line and start it. Then we have the OCCI interface. We have an implementation for OCCI in OpenStack, which is called OOI. OCCI, for those that don't know about it, this is a RESTful protocol and API that is focusing on cloud interoperability. So it's mainly focused on infrastructure as a service, but it's accessible to other areas. This is a standard defined by the OEF. And what we have done is to deliver a new implementation of this OCCI standard in OpenStack. It's completed within Frontend Scratch. There was another implementation some time ago, but it was not properly maintained and had some issues with using internal OpenStack API. So we decided to start from scratch using only public APIs. So it's easier to transition from one OpenStack version to another. It supports VM management, volume management, and network operations. And can be installed easily with a NOVA API endpoint or as a separate whiskey application. So in the global OCCI picture, we have OCCI has a set of client tools. We provide Java APIs, Ruby APIs, common line interface. And you could have any other OCCI independent client. We have a version, which is in the middle, this Rocky server that is used right now in OpenNebula, but also can talk to Azure and Amazon web services. So it translates OCCI to those APIs. And we have then OY here as a layer that translates OCCI to OpenStack. So a client with OCCI support could go into any of these resources using the same API. So this is everything I have talking to you now in a single picture. Let's say how we integrate everything with OpenStack. And well, you can see all of the blue boxes are what we are providing as a components that you can put into your OpenStack to integrate with EGI. Then we have gray boxes would be the vanilla OpenStack services. Users would just get tokens as usual, but using this BOMS thing. And then they can interact with the NOVA API or the OCCI APIs. And then we have a set of EGI services that also interact with the site, the resource provider. So we have the monitoring that checks that everything is working. And we have the accounting repository that collects information about the sites. We have the BDII to collect information and this image subscription with app DB. This will turn soon into using the EGI chicken. So I mean, very similar to the previous slide, just changing these Keystone BOMS to OS federation extension of Keystone. So it would be just upstream Keystone no modifications there. For the other blue boxes, we provide the appliance, which is a VM that you can start in your cloud and configure. So you will talk to your public APIs and do all the things for you. So you don't have to worry too much about internal details. All of the components there are available as Docker containers. So you can also run it with Docker. We have full documentation about this and the appliance is published also in the app DB. So you can get it from there. And of course, you can use tech technology that we provide to build your own custom federation. Maybe you find some of these components interesting for you and you want to build your own federation. What we are proposing in EGI is that if you want to have a cloud realm with us, you should follow the mandatory things here. So you should follow the EGI federated service management, which means you need to be compliant with the policies of access, follow security coordination that we have. So if there is an issue, you have to respond to it, et cetera. Then you should register your services in the EGI service registry. You should comply with the EGI AI and also provide accounting records to the EGI account repository. Monitoring, once you register anything in GoDB, it will be done for you. So this is mandatory, but nothing to do from the provider side. And then we have the other services which we consider optional. So if you want to build a new cloud realm for your community and you don't need the information discovery, that's fine. You can do it like that. The same thing with replication of virtual machine images or integration with AppDB or using OCCI or using the EGI helpdesk that we have. So we provide the technology, and if you find it useful, yes, please go ahead and use it. And if you want to have a realm in EGI, please go ahead and get in touch with us so we can create a new one. Now I have a few examples of who's using Cloud Compute right now. It's a very high level view. We have this web page in the EGI website with details of use cases. So for example, I have six. So different areas. One is from the Astro Astronomy area. So that's the extras project that is collecting a lot of data from 13 years. And it's using the Cloud Compute to implement the analysis of this data. Then we have the dream which is about Earth observation. So they are simulating hydro-mentological events, such as floating, et cetera. Then we have another use case from Sweden, which they have a set of tools for doing dynamic analysis. And this is quite use worldwide. So it says more than 6,000 users in 73 countries. And this is an application that you just put in your laptop and connects to the cloud to do the calculations. Some other examples also in the bio area, this example that was published in Nature and not so long ago about Salmonella and doing some RNA analysis of the bacteria. Then the cloud of EGI has been used to do simulation of seismic events, like the earthquake that happened in Italy this summer. Or another example is pitch note, which is about analyzing music. So they also run their analysis software in the EGI cloud. So far, we have that. We want to improve our federation. First, we want to improve the user experience by providing this certificate less access with new EGIAAI. We are extending this up-to-be marketplace with a new feature to be able to do the VM management directly on the web page. So you have everything in a single place. We have a new version of the OCCI standard coming that is improved. It's much easier to implement and has better networking support. We want to go beyond infrastructure as a service and want to exploit results from one of our sister projects, which is the Indigo project to offer platforms as a service for our users. And we also want to get more involved in OpenStack. So we want to get more involved in the scientific working group and work into identity federation, which is something that we need. And we'll try to push for changes where we need them. I have some references here. This is our web page. We have a section of our wiki with lots of information about the federation. We have quite detailed installation manual. So if you want to get technical, you can really get technical. And we have a mailing list, which is this one, where you can send your mail, participate, and have discussions there about everything, about the clouds and research. So thank you very much for your attention. There is some time for questions. So if you have some, go ahead. So in your talk, you described how you're doing authentication on the front end. You're going from the old ProxySert to the new method. But you also described a service registry and then also the app marketplace. So given when a user authenticates and gets a set of attributes, do you use those to limit or control what kind of service discovery that they do and also what kind of app discovery that they do? You had the example of the Biomed, I guess, group or something like that. So when the user authenticates, do they get a Biomed attribute? Are they a member? How does that work? So this EGI check-in will be integrated across all of the EGI services. So the service registry, the GOGDB, is already integrated. So if you log in with the EGI check-in, EGI check-in will return to the GOGDB. You're a member of this VO, or you have an admin role, which is mostly the most important thing in GOGDB. You are owner of this resource. So you can edit the resource. You can add new endpoints there. So everything uses the attributes from the EGI check-in to do authorization. And different members of the Federation different sites, can they register different service endpoints in the service catalog? So in the service catalog, you have an owner of the resource center. And that owner can add new people there. So you have resource center and means. And that information is stored there. And for the group membership, we are using BOMS right now. But we are able to plug into other attribute authorities that may appear. And we think that there will be more than just BOMS. And that information is also included in the claims. So the app DB or GOGDB, anything, can check those claims and present a restricted view or anything you want to do. OK. And then also, in terms of how you manage your VOs, is there, I mean, there is a BOMS. There is a VO management system somewhere. But I didn't see that described here in any detail. So maybe I can go there to the far away. So BOMS would be this attribute authority here. So right now, as attribute authority, we are using BOMS. But DCGI checking is able to use anything with a rest interface. OK. In terms of your attribute authority, is that where VO attributes would reside? OK. So in terms of defining what's in a VO or associating VO attributes with a specific user, that's where it would be done. Yeah. OK. You can have more than one attribute authority. So it's not restricted to just one. You could have as many as you want. And you would have to have some sort of consistency between them? Yeah, we have defined. There is another, let's say, sister project called AARC, which is all about authentication and authorization, where we are trying to standardize how these claims are defined. So everyone has the same staff and the sites, the resource center can know what to expect. So they know the claims will come in this format and will know how to deal with them. OK. Thank you. You're welcome. Any other questions? Can you go to the, because otherwise it would. I would actually go. OK. In your very first slide, you mentioned, or maybe the second one, that you provided different sources of resources, heart, report, general purpose, training, that sort of thing. I was wondering how you manage, I guess, the capacity planning around those things? So we are actually doing that exercise to define how we are going to do the capacity planning of everything. So I'm not involved in that area, so I can give you a good answer. We are working on defining the process for the capacity planning. So yes, we are aware we need to do capacity planning, and we are defining how to do it. And it will be done soon, because we are standardizing all our internal processes. But right now, I cannot say, OK, we do it like this or like that. But yeah, we are aware that we need this capacity planning. So I think that's all. Thank you very much for your attendance.