 Hello everyone, welcome to this demo that will be about community driven knowledge about Python packages and we will talk about recommendations that TOT is the recommender system provides and also we will talk about prescriptions concept and what information we keep in prescriptions and how they are used during the resolution process in cloud based resolver that is TOT. Okay so just to recap our mission is to help developers with application development and we would like to provide a way to help developers ship better software. What better means it can mean better when it comes to bugs but also better performance, more secure software and things like that. So to support this idea we created a community driven database that is in a form of YAML files and these YAML files are automatically consumed by the cloud based resolver and are used during the resolution process to adjust the resolution of Python packages so that they are well performing secure bug free or there is provided some additional information and additional guidance for Python developers. So let's have a look at the current knowledge that is used during recommendations and what we provide to users. So as of now users of TOT can ask for an advice and the advice will give some information to users so just to list information that we provide we provide a URL to PyPI to the corresponding Python package release so it's specific to version so if you consume Python packages the recommender system will point you directly to the release that is installed and recommend it. The other information that we provide to users is URL to GitHub repository if the system detected that package is hosted on GitHub then it provides a link directly a link to GitHub repository so you can browse sources you can browse content and also the system will warn you if a Python package is marked or the repository is archived so that's another information that is stated to users. Then when it comes to GitHub we also link to GitHub release notes so if the system detects that there are release notes on GitHub for a specific Python release then the system points you to these release notes so you can browse possibly backwards incompatible changes or what is new in the new release. Then we also detect community on Stack Overflow so based on tags that are published by Stack Overflow we associated tags with Python packages so if you consume Python packages that have Stack Overflow community the recommender system will point you to the stack to the relevant Stack Overflow threads. Then we link also information to libraries IO that is a page that accumulates some information about open source projects and open source Python packages so we provide a link to libraries IO we also provide a link to pulp that is let's say secondary source of Python packages that can be released by Red Hat teams so you can also browse content on pulp we also provide information whether a Python package is packaged as an RPM in distributions so as of now we provide this information for Federal Authority 4 and UBI 8. Additionally we accumulated some knowledge about Python packages more precisely we accumulated for example issues that are known so resolver adjust the resolution resolution process so that the resolved software stack does not is not resolved with for example bug buggy release an example can be this error so you can see the ammo file this is these are the prescriptions and here you can see that if pillow is installed with numpy pillow in some specific version then users get some errors you can browse upstream issue or upstream issue tracker to get more information about this so you see there is a runtime error and the suggestion is to use different Python sorry pillow release so that's one type of guidance that we give to to users okay so now some of this information is derived from our main knowledge graph but we try to keep the database open and that means we created this declarative interface for the resolver in the form of ammo files that are managed so and released so the database of these ammo files is properly versioned and released to deployments and in that case the resolver automatically consumes consumes these so-called prescriptions and can give additional guidance to users or adjust the resolution process in a way the resolved software stack is or the result software stack has high quality so as of now we have something like something more than 5500 the ammo files of these prescriptions that is roughly 41 megabytes and this database it can grow we will be happy if it grows and if you have any issue in your application stack or you know about some issues feel free to extend this database feel free to contribute to this database so we provide better Python ecosystem and better libraries to users so users or developers do not have headaches with debugging code and rather focus on delivering features and delivering applications okay so let's go to terminal and let's have a quick demo so here i'm having a very simple application hello pi and let's have a look at dependencies that are used in this application so here you can see direct dependencies of application and let's go to tot so that tot resolves these dependencies and can give us guidance so if you are familiar with tot you already know that one of the integration points of tot is a CLI tool that is called Tamos and you can issue Tamos advice so in that case the direct dependencies together with some additional information such as information about runtime environments, Python interpreter version, operating system, base container image but also static source code analysis of your application this all is sent to the backend input is for example recommendation type that states your intention with the application and all this information is accumulated in the request and sent to the backend the backend then asynchronously resolves your software stack based on the knowledge that is provided so as we've seen knowledge in these prescriptions but also knowledge about dependency structures, dependency structures, structure or other information that is stored in the main knowledge database once we obtain results from the resolver we have two main parts the first one is application stack guidance that is guidance on application stack and its general guidance and also information about runtime environment and things like that so you see that I'm not using tot's S2I that is the recommended base container image that is analyzed by tot and can give better well and when this tot S2I is used users can get better recommendations so it's recommended to use tot's base container images then also some information about my configuration so I did not configure for example platform so there is implicitly supplied linux x8664 then you can see also information about prescription release so you know which database of known issues is used and where this database is hosted then information about missing cpu model cpu family in my configuration file that means that for example the recommender system is not capable of giving me better guidance when it comes to performance when it comes to performance of my application stack then we see these warnings that are saying that package is removed from the resolution process basically it means that the system based on the pre-aggregated knowledge based on pre-aggregated knowledge from analyzers that are run on the background spotted that some packages failed to install into my runtime environment into the environment that I'm using so for example that can be caused by a python interpreter version that I'm using some packages are compatible just with python 2 and they can cause installation errors installation errors when python 3 is used and similar errors okay so this is application stack guidance part you can possibly spot more things such as rules so dot operator or the person that operates dot can configure rules on the back end and these rules can avoid resolution of some specific packages so here is a rule that avoids using very old packages so for example beautiful soap in version 4.3.0 was released before December 2016 and that is considered to be old release so users should not use it or there is another reason that total operator can state when configuring these rules okay so that was the first part of the result or of the information provided by the recommender system and when we scroll down this can be pretty verbose given how large your stack is and what packages you use the second part of reports that is provided by the recommender system is the recommended stack report in this case this report is specific to python packages that you consume so you can see information that I noted earlier in the presentation you can see information about for example link to libraries IO so I'm using ArcPars package so the recommender system points me to libraries IO where information about ArcPars can be seen so if you are interested in data that libraries IO accumulate you can browse that another link points me to the specific release on pipi so here you can see ArcPars in version 1.3.0 that is the recommended version by the recommender system so here you can browse meta package metadata that library maintainers provided for project read me that was used when packaging ArcPars and publishing it to pipi another information is derived from metadata from package metadata so in this case system extracts some metadata from python packages such metadata can be true classifiers that are used in python to classify packages so in this case I'm using python interpreter in version 3.6 but the system detected that ArcPars that I'm using or that was recommended was not tested or was not released with python 3.6 in mind so if we go to pipi.org we can see some metadata that is associated with the package and here you can see a list of python interpreters and python interpreter versions that were claimed to be compatible with the the ArcPars release okay if you would like to know more about these truth classifiers there are more recommendations that the system can compute follow demode this specific to truth classifiers another recommendation or another link is warning me that the package that I'm consuming is is archived on on github so it gives me a link to the repository where ArcPars is hosted and as you can see the repository is archived that means you cannot open new issues you cannot open new pull requests and that means that development of the given package is that so you should consider using another package or another project with similar functionality because this package is no longer maintained and has inactive community another recommendation is pointing me to libraries IO specifically for beautiful soap 4 also version or release on pipi the system also detected that beautiful soap 4 has active community on stack overflow so it points me to threats that are specific to beautiful soap and in that case I can browse knowledge that was accumulated by the community on stack overflow or I can find issues that are known in the community again beautiful soap in that specific release in does not have Python 36 truth modifier then another information is specific to pick another package that is dusk moreover the system points me to documentation of dusk dusk can requires additional configuration when using so system points me to document documentation how I should set up dusk and how I should use it then another package or another information is about decorator so libraries IO a link to pipi but in this case the system detected that package decorator is also packaged as an RPM in UBI 8 and I can find it as Python 3 decorator RPM similarly for network X I receive a link to libraries IO pipi a link to repository the system detected that network X hosted on on github where I can find community where I can find sources it also points me to documentation it network X has active community on stack overflow so I can browse browse threats on stack overflow okay so this was regarding this type of information now I'm looking into the log if there is something more worth stating maybe maybe release notes so if you are consuming a python package and you would like to know what has changed across releases or new features that are provided or release notes in general you would like to browse release notes in general then the system detects where packages are hosted and where these release notes can be browsed so for example I can browse a sci-fi release notes these release notes are specific to sci-fi 1.5.2 and I can see anything that by package maintainers of sci-fi published there okay so that was the demo if you are more interested in thought feel free to visit us we are a community where you can find more information how to reach out to us where you can find us in at totstation.ninja so feel free to go there if you are more interested in prescriptions you can find prescriptions in github repository that is called prescriptions there is also a separate demo that browse the content of prescriptions and if you would like to know more about prescriptions or write prescriptions you can browse documentation of these prescriptions so here you can find totstation slash prescriptions repository you can browse these database so the database was created out of python packages that are packaged as rpms in ubi-8 or federa 34 but also this knowledge was accumulated based on python packages that are highest rated or most downloaded from ipi so top 5000 python packages was used to accumulate this logic so for example you can browse i don't know packages that start with te and you can find packages such as or prescriptions specific to tensorflow package where you can find release notes link and other prescriptions in this case for healing dependencies so this way i would like to thank you and see you next time