 Yes, so hello everybody. This is Mike Fedosin speaking, but before I begin just a couple of words about us I'm a Glenn score member philosopher doctor in technical sciences and a passionate programmer with more than 10 years experience in developing of web services My name is Karat Kushaev. I was involved previously in heat and glens projects right now. I'm developing glare project I has been involved in developing in open-stack upstream for two years Okay, so today's session will be dedicated to a new service in open-stack called glare Actually, this service is not new because it has been incubated in Glens repo for more than a year as far as I remember from liberty cycle But now it's absolutely stand-alone project. It has own repo own client and even own spec repo But actually there are plans to bring it back under glance governance again to make both projects have benefits from each other and By the way, the name glare came from Glens artifact repository So what is glare? glare is a service that provides a unified Catalog of binary data along with its metadata such structures are often called artifacts We wanted to respond to community needs and create a unified repository of artifacts for all open-stack projects Because currently in open-stack, there is no single solution in this regard some projects store their files in databases on github Directly in Swift for Nova. There is a service Glens and so on Nevertheless, none of these solutions do not eliminate issues of artifact versioning and data immutability and of course do not offer convenient means of storing and searching a key feature of glare is the ability to manage different artifacts of various nature from Nova images and docker containers to hitting plates and Database backups at the same time it doesn't matter what artifact type you use glare always provides a unified interface for working with them This flexibility is achieved through the plugins operators or users can independently describe their own artifact type and connect it to glare after that glare will take over the entire process of Handling this artifact type and its connection to the system to define new artifact type We use slightly modified library Oslo versioned objects which already gained its popularity in open-stack and for this reason I think it shouldn't be very hard to implement new artifact type because it's done in Python and in familiar environment Moreover currently Several artifact types are already included in glare. We call them blessed artifact types they are images hitting plates hit environments more under packages and Tosca templates and soon we are going to add torrents for ironic project Okay, so now let me describe other useful Glare features first of all it is artifact versioning which means that artifact is supplemented in with the version in a Samware format and user is able to sort and search by version Then we have special Dynamic filter for that. It's called latest which literally says give me artifacts with the latest versions only Then the supports several binary objects or blobs per one artifact From Glare's part it allows to specify a set of such attributes So for example, you can create new artifact tape Amazon image and Specify three binary object there its kernel RAM disk and memory and then access them directly If the number of similar files is unknown like it's done for example In OVF OVA format or in VDI you can specify a folder for them in glare. We call it blob dictionary and Put all your files inside For metadata glare support structured properties dictionary and listen you can specify things like dictionary of strings or list of integers like everything Then we have this conception of dependencies and links Which means that under the fact may depend on the other It's a very important feature and we have sort of slides dedicate directly to this and of course We have advanced filtering with various operators like not equal greater than and so Now let's see how glare is organized from technical point of view and I pass the mic to my teammate Karat and he will tell you about glare implementation So let's start from The basis. What is artifact we define artifact as some user entity represented this artifact type artifact just second Okay, let's start from a basis We introduce the term artifact as some user entity Represented as artifact type it can be glance image heat template heat environment or everything else Each artifact type defines which properties are allowed for this entity for heat templates It can be environment template template version and so on and of course glasses have glare itself Introduce some typical actions allowed for each artifact like create upload download share or delete this artifact The properties allowed for each artifact type can be split into main groups Common properties best properties are presented in each artifact type It could be ID name version creation date and a lot of our properties like status and so on and type specific properties As you may guess, it's just Properties exclusive to type like for heated heat template file Template version for Tosco template. It's Tosco template file for glance image image version in format Each property has own metadata define it in glare itself. It's We introduce some metadata about properties such as name Default value for each property and of course type We support most of Python types as artifact properties But we also introduce a two additional types with makes a difference. The first type is link it documents dependency between two artifacts and You can consider it as a sim links in Linux. So when you delete referenced artifact nothing happens The second type is a blob property. This is the key feature of glare It allows to manage binary data Using glare artifact. So let's talk about it in details Currently we are two approaches to manage blob properties The first approach is upload the data directly into glare in this case You for example provide some image file or heat template and this template will be uploaded to glare backend After that all left cycle will be covered by glare For example when you delete heat template artifact the reference blob file in heat in glare backend will be also deleted We also calculate some checksums like md5 and ours for each blob The second approach is just to specify some external link instead of blob It is applicable in some cases when you have File on external HTTP server outside glare and in this case It's just text reference in glare and glare doesn't cover or life cycle cases for this blob Additionally user must specify md5 and our checksums manually for this file So here you can see the example of definition for artifact. It's a real example currently Implemented in master. It's heat template. You can see all fields like environment template file itself Which is blob list of nested templates and default environments This was environments will be Applied to heat template where when user doesn't specify any our environments As you can see there is no such common properties like you expected ID name and version West properties defined in base artifact, which is parent of this class Here you specify this specify ID name and our properties They also define defaults and some filters and our metadata We are going to prepare some documentation about how to specify process and we will cover all available options in this document Okay, now we have some Notion about Definition of artifact what we what can we do with this artifact and glare The typical workflow For glare artifact is we following first we start with artifact draft It's just some draft version for example of heat template After that you can update some properties Required for is this artifact for example for heat template you must specify template file without this template file The artifact itself is useless so after definition of all required properties and block values Artifact is ready for activation Once it activated you can share the artifact with our users so you can make it public Of course, we support some additional actions like delete or activate and so on it's mostly based on glance workflow for images The activation and publishing is the key steps in artifact workflow in artifact lifecycle Activation Allows user to tell the system that artifact is ready for production. So Everybody can start using it When artifact is activated the system checks that all required properties are set for example for glance image image file must be specified before activation And all properties after activation becomes immutable So no changes to image after activation no changes to template after activation no changes to toss contemplate after activation Where is some exception? Mutable properties because in some cases we need to specify properties after activation one example is text We need to specify text after activation because we can be changed So you can explicitly mark is it as mutable it and it can be changed after activation Activated artifact as I mentioned before can be published Publishing artifact Makes an artifact available for all users in read-only mode so we can download this artifact get or list By default publishing is available for admins only, but it is managed by policy so can be changed if we So if artifact is immutable how can we for example change Introduce a change for this artifact for example if we have hit template and some properties are changed We need somehow deliver this update to our users To support this case we implemented versioning versioning allows User to ensure that there will be only one artifact with specific type name and version We introduced several scopes where the artifact must be unique The first scope is private It means that you can't create an artifact with the same name type and version within a keystone project This second scope is public So you cannot create any public artifact with the same name type version If there is some contradiction between private and public then private is Returned first so it's like in programming local over global If you consider all these features useful then perhaps you Want to use all this feature for your custom types glare also provides this possibility You can write some Python class Put the reference to this class in our config and after that this artifact will be available in your environment For example, you can see this type you just need to inherit from base artifact introduce some fields and Also, we implemented some Like a hook functionality that allows you to validate data before Uploading before publishing before activation. For example, one useful use case is Tosca We investigated that we can we can pass the template To Tosca parser and validate it before uploading so we can ensure that all Tosca templates in our environment will be valid from parser point of view Additionally, we implemented a lot of our features like micro versioning We also support different backends because a part of our system is glance store so we can store your blobs in Swift save or local file system, but it's only for testing and We also introduced some discovery functionality that allows you for example Just put the reference to your class and after that we generate which is on the definition for this class automatically and Well, last but not least we introduced some additional admin admin artifact type It returns all artifact type despite all artifacts despite its artifact type So it's very useful for admins who doesn't bother about specific artifact type and can observe All available artifacts in the system So I pass my So okay folks now let me demonstrate to you a couple of examples How glare can be used in your cloud? The first example is Application catalogue based on community app catalogue. The second is a storage of hitter plates and heat environments So community app catalogue It's a website that contains various artifacts for your cloud currently. There are four types There are images hit in place to us contemplates and run the packages in the new version It's based on glare which handles all inner operations on the artifacts and all artifacts are publicly available There which mean that each person is able to download them without registration Nevertheless the creation process requires Anauthorization it it consists from three steps First of all user has to decide artifact of what type he is going to create let it be Tosca template for example and Then initially he has to specify just name and unique artifact version After that glare will create an instance in database in drafted status and Catalog will generate a web form that contains various artifact fields like description author name tags and so on User may fill them or leave empty and finally He needs to upload Tosca template file as carat said we have Support of data validation uploaded data validation for Tosca template It's based on to Scaparser and it validates if uploaded file is a very Tosca template if so glare puts the file in Swift storage or safe whatever and Returns okay code otherwise if upload failed then glare returns bad request and user will have to fix his file and upload it again And by the way all web forms are generated automatically From Jason schema for from Glare service as carat said again Glare is able to create Jason schema from artifact description and for this reason Catalog administrator doesn't need to create new UI if he wants to add new artifact type like docker containers Catalog will do it automatically for him. The second step when uploaded upload is done user activates the artifact which means that this artifact is completed and can be used in production Glare verify that all are required attributes were filled for Tosca templates user just needs to upload a File but for example for images. He also must field disk format and container format fields if everything is fine glare changes the status of the artifact to active and Sends a notification to catalogue administrator that this artifact may be published and on the third step catalogue administrator may manually check the artifact before publishing and If he's pretty sure that this artifact is good enough He changes the visibility to public and this and you just contemplate becomes available for everyone after publication if admin considers that this artifact may be harmful He may deactivate it for some period of time and do some additional instigations If everything is good, he activates it and it becomes available again. If not Here he moves it completely So the second example is a storage of heat templates and heat environments The main thing here that the sentries are useless separately and only their combination will do the trick and Another point is that heat requires several nested Templates for one artifact and glare should support it somehow. So let's see how it's done First of all for nested templates. We have a special folder what dictionary called nested templates and user is able to put all his files inside of it and then access them directly by their names Then heat template artifact type has a dictionary of links to environments That can be used with this current template and also we have special Dictionary of strings called default amps which contains names of environments that can be Used with current template if nothing were specified by user So tomorrow there will be Another session provided by Alexander Tbilkov and Kirill Zaitsev As far as I remember it's called something like an understanding of glare ap catalogue and Murano So I believe there will be more examples and if you are interested you can come there and listen So finally, let me Describe other features. We want to implement in glare in future. First of all, it is artifact sharing We have been implementing our sharing model from the very beginning and initially We wanted to have it like in glance v2, but then when the student has several flaws Actually, there are six use cases and two questions How artifact may be shared and with whom artifact may be shared How there are two options accept and no accept the difference between them is that In case of accept user has accept the sharing before he is able to use the artifact For the second case artifact will appear in his output if he wants it or not So if you are familiar with glance v2, you understand that I'm talking about shared and public Visibilities Then there are three options with whom artifact may be shared. It is some particular user some role and finally all users and We are going to support all these scenarios cases in glare in glance Only two currently are supported. It is accept user and no accept all and as far as I know They're implementing community visibility which actually is Accept all but I think it is a little bit redundant because it's a very special case of shared visibility So if you want to know more about implementation, you can come to Glance meet up on Friday and I'll try to explain the solution more detailed Then it is dependencies and links. I Think it's a killer fission glare because it allows to combine several independent artifacts in one structure with varying flexibility And in glare we are going to support three different types of relations dynamic soft and hard soft are currently implemented We call it link then dynamic they are dynamic links and hard dependencies are Hard is called dependencies For dynamic links user has to specify some search query and Then glare limits that put only on artifact and return it to user for both Links and dependencies user has to specify concrete artifact ID Though the differences between them are on the slide in case of dynamic links user can refer to various artifact in different time intervals both links and Dynamic links can refer to an artifact in another cloud so can they can be external For dependencies glare verifies that the dependence artifact won't be removed from the system For links and dynamic links there is no grantee on that matter and of course user must be an owner of the artifact to set dependency on it and also for links and dependencies Glare verifies that the dependence artifact exists when this link or dependency is set Other features are it is import and export of artifact because sometimes it's required to have the whole artifact as Just one file as you know artifact consist of several binary objects Blobs and various metadata and We are going to implement special type format dot glr Which contains all artifact contents and it can be used as independent entity it will be possible to import your artifact from cloud a then save it on your flash drive and Expert it on cloud be with no differences and in the future it opens away to glare-to-glare communication When some glare services are playing role of proxies for other glares then Documentation is our weak point. I know and we are going to improve it dramatically Artifact signing and verification is on the map, too But there is not special we are going to use the same model like it's done for Glance images And I think it's important feature because it improves the stability of the system and did it allows to provide accurate data to the clients and Also, we are thinking about adding new blessed artifact types Currently there are five I mentioned them soon. We're going to add ironic and also There may be docker containers and puppets and Ansible playbooks So there are contents contacts how you can participate in glare development first of all, it is our IRC channel and To repose on github it is just to report and it's client So in conclusion, I want to thank all people who were involved in the process of creation of glare Especially Glance community, which was very generous and helpful and of course my teammates Karat, Sergey and Darya Thank you for listening. So any questions? Thank you so This project seems to be Somewhat competing with Glance in a way kind of and my question is whether it's actually competing or this should be just like an addition to me it's would be seem more natural to have it like extension of Glance or Glance version 3 which is Probably is now renamed to glare. Could you clarify the strategy a bit more? I? Think we should work together with Glance community as and as I mentioned in the beginning of this Presentation there are plans to bring glare under glance governance again. I think these projects are work good together and Yeah, it can be additional service for Glance, but we need to decide it and this Glance community from Technical point of view glare allows to do the same things as Glance We don't have metadata, but they are not actually part of Image API as far as I understand So it I think it should be some political decision Mostly Glance is Concentrating on image images itself with an opposite community, but we are concentrating on artifacts some stores versioning stores Of course We will be some competing feature But once we will be major we will consider it may be combine or split features between these two projects Yeah, I would just add to that because sometimes like in Glance once you run your cloud long enough You will have a need to version your images and that's where this fits actually and For example, at least that feature would be really nice to see in Glance Signing and all that stuff would be also beneficial as well from first look and So I would just have one more question that would be about back-end support. You said you will support multiple back-ends Will they be available at the same time? And will be will I be able to use for example to save clusters as two different back-ends at the same time? We use the same library as Glance. It's called Glance Store. So there. Yeah, there are possibility to use several Backends at the same time, but there will there will be only one default Where where we'll put all files, but then it can download it from CF from swift from HTTP locations and so on So it's absolutely like in Glance because we use the same library one active where you can create and our Like like a passive where we can download with artifact from us back-ends Okay, yeah, thank you, but we also open to new features So if you need if we really have a strong use case for this, we are open to it So yeah, please contact you can consider this because you know, we are no new project and If there will be strong use case for support multiple back-ends So maybe we need to reconsider our architecture at this moment. So, yeah, okay, I'll try to attend Glance sessions later and Glare as well. Thank you Yeah, so a bit of a general question, maybe So I'm wondering how how do end users consume the artifacts? So for example, if if you have heat templates stored in glare Is it like the end user needs to retrieve the template that they're interested in and then push it to heat? Or is there gonna be some connection between? So Yeah, there is a project and the initiative on heat size that will add support of glare directly into heat I don't know in what state right now. It is but actually it is From where parts You work with heating plates Through urls, so you have an url for hitting plate artifact and then you put your for your files inside of it and then download them so Upload file it just put request there download it's it's get request Yeah, so yeah, we lack of integration at the current moment there are plus certain plans about the integration between heat and From heat and morana team. I think it will be very useful to have Just heat to return with template from glare and upload it in no local files anymore Yeah, the same is applicable for morana package for any artifact. Yeah. Yeah So we are open to it It just need to implement it on our company site and yeah If you need to use it if you we have some guys from companies, please contact us We are open to new integration to new cases. Okay. Thank you. So it seems no more question. Okay, so thank you very much