 All right, so without any further ado, we have Frido presenting Thought Healing Python Applications. Take it away. Thank you. So hello, everyone, welcome to this presentation that will be a set about Healing Python Applications. And in this talk, we will talk about things that are developed in Thought. So let's get started. Before we start, let me introduce myself. My name is Frido Lim. You can find me as under acronym Fridox or on Twitter or on GitHub. And I am a Thought member since 2019. So that's basically from the beginning of the project. I like Python. Actually, I love Python, and I do road cycling. Now, when it comes to Thought, that was a research project that started in AICOV team in the office of the CTO. And one of the main offerings we do is cloud-based Python resolver. You can visit us on Thought Station Ninja web page where you can browse documentation, but also links to YouTube channel where you can find our demos or Twitter where we post periodic updates. So as stated, one of the main offerings we have is a Python resolver that is run in the cloud. Basically, it's not just a resolver, but it's a recommendation engine for Python applications. And we will talk about main features and why the recommendation engine can be suitable for developers. The recommendation engine is publicly available to the community, so you can use it. And the core of this recommendation engine is a stochastic resolver that is implementing gradient-free reinforcement learning methods. In production, we run temporal difference learning and that's something we will talk about more in this presentation. If you are interested in details how the core mechanics and how the resolver is designed, you can browse documentation that is available online. OK, so if you've ever used Python, you probably know that's the Python installer that can install dependencies into your environment. So if you have a dependency, you simply install it and it's downloaded the resolution process, also results, dependencies that are needed to install the library and then you can use the library. Now, what we do, we offer an alternative that is a CLI tool called Tamos. And you can install Tamos using PIP. And once you have Tamos in your environment, then Tamos can generate a configuration file for you that is subsequently used in the cloud resolver. So if you want to give Todd a try, you simply can issue PIP install Tamos, Tamos config that will generate a configuration file and then you can do Tamos advice that will take the configuration file and together with requirements of your application, we'll send them to the cloud resolver. Now, why the configuration file is needed? As the resolution process is not done locally, as in case of PIP, the backend resolver needs to know how to behave in this resolution. So the configurations file states information such as operating system that you use, Python interpreter version that you use, but also other aspects that are not considered in PIP such as CUDA version that you have available, but also base container image that you are running. So the resolution process is really specific to your runtime environment. Besides software information that is captured in Tamos config, Tamos also captures information about hardware that is available on your host. So it captures information about CPU and GPU and this is valuable to the resolver because if the resolver is targeting performance applications that we will talk about later, then the resolver can take into account also hardware information that is available on the host. Besides this information, the Tamos config also can optionally gather information about source code for which the recommendation is computed. So Tamos config, sorry, Tamos advice under the hood performance static source code analysis and extracts information such as what libraries are used in your sources as well as what symbols from these Python libraries are used. Then there's one additional option that is recommendation type and this recommendation type basically says intention with the application that you have. So if you are developing an application that should be secure, should not have any security vulnerabilities, then the resolution process can look differently in opposites to resolution process that is targeting very well performing software used for training machine learning models, for example. So this is all inputs to the cloud-based resolver and the cloud-based resolver uses this input together with knowledge that is aggregated about Python packages but also about runtime environments and can resolve a software packages so that it provides log file together with some justification that says why the given log file with set of packages should be used. As stated before, the recommendation engine uses reinforcement learning. This feature or this implementation of reinforcement learning was brought to the resolver to come up with the best or the most suitable set of libraries that should be installed into your environment. If you consider all these inputs that are going to the resolver and all the possible versions that are available out there, versions of Python libraries, but versions of CUDA, versions of operating systems and stuff like that, then you can imagine that there's a wide range of possible resolutions and all depend on requirements on your software environment and on your hardware. So there's possibly infinite state space of all the possible resolutions. So the resolver needs to somehow know how to navigate in this state space and how to resolve the most and the best possible software libraries that you can use. On each request, we train a model. So the resolution, each time you submit the request to resolver, the resolution trains a model and the resolver first performs exploration phase where when the resolver checks what are the possibilities, how it can resolve software packages and subsequently resolver performs exploitation phase. So based on knowledge that it aggregated in the exploration phase, then the resolver can come up with the resolved software that is considered the most suitable for your runtime environment where your Python application runs in. The whole resolution process is then treated as a resolution pipeline and this pipeline is made out of units of different type. There are core or base pipeline unit types that are registered in the resolution process and then you can also extend the resolution process by implementing your own pipeline units that are of specific type and the type distinguishes when these pipeline units are called and also semantics behind these pipeline units. The implementation of the recommendation engine can provide a way to extend the resolution process by implementing these pipeline units and this extension can be done error in Python code or you can declaratively provide the ammo files that are automatically taken into account in deployment loaded by the recommendation engine and can adjust the resolution process in a way that the most suitable packages are resolved. The pipeline is again constructed dynamically so on each request pipeline is constructed based on inputs to the resolver and also based on the knowledge that is aggregated in the Python ecosystem. Now let's focus on the declarative interface for the resolver and let's check these ammo files how they are written and how they can be used. These ammo files are called prescriptions and they prescribe some rules how to heal Python applications. So these prescriptions form a declarative interface to the cloud resolver. If you are familiar with OpenShift or Kubernetes you have manifest files that you apply to the cluster and these manifest files basically desired state of the cluster. Similarly these prescriptions can be applied to the resolver but in this case they somehow adjust the resolution process in a way that resolver comes with the most suitable software packages that you can use in your application. As stated before these ammo files are automatically consumed by resolver in a deployment and our versions so we do releases of these ammo files as well. If you are interested in prescription concept and you would like to write your prescriptions feel free to browse the documentation that is available on our homepage. Now let's take a look at some examples. So these prescriptions were designed to help resolving high quality software. That means that sometimes some releases create some runtime errors or undesired behavior that library maintainers did not test it or simply there is some misbehavior in libraries. So here you can see the very first example in which case pillow inversion 830 does not work with any NumPy. So if you install pillow together with NumPy pillow in the specific version you will always get this type error that is raised on runtime. The issue was reported on upstream and newer releases of pillow do not have this issue. However, if you need to stick with older version of pillow for some reason this issue can arise. Note also that even if this issue was seen in older versions of pillow that doesn't mean that the resolver always comes with the newest pillow. The version that is resolved depends also on other dependencies. So if there are shared subgraphs in dependency graph dependency resolvers can come up with pillow 830 that will cause these runtime errors in your applications. So to bypass this we can create a prescription in a tot that will avoid installing pillow together with NumPy. So here you can see a YAML file that states the prescription. As you can see it is created out of multiple sections. So the very first section states units that are declared in the specific prescription. In this case we are using pipeline unit that is of type step. So this is specific to a step that is performed during the resolution process. And the name of this pipeline unit is pillow 830 type error step. This name is used in the resolution process to uniquely identify this pipeline unit in logs or when checking prescription schema. Now the next section says should include which has just one directive and it is advisor pipeline true which basically says that this prescription is used during recommendations. In the upcoming examples we will see other declaratives that can be stated in the should include section and they additionally configure when the given pipeline unit should be present in the resolution. The next part of this step pipeline unit says when the pipeline unit run should be triggered. This is the match section that says the pipeline unit should be run when pillow in the specific version coming from by PI org is about to be resolved and resolver has already one dependency that is resolved and that is numpy. This is not specific to any numpy version as can be seen and also not specific to any numpy coming from different index than by PI org. Now the final section that is run states how the pipeline unit should behave when the run is actually triggered. So here you can see that pipeline unit does not accept the step that is performed by the resolver. So there you can see not acceptable pillow inversion 830 does not work with the given numpy. And then there is also provided some information to the user that this pipeline unit discarded the given step. So users of talk can see this in the resolution block. Okay, so now let's take a look at another example that is adjusting requirements in GPU enabled environments. If you are not familiar with TensorFlow, TensorFlow is a machine learning library and its dependency graph is quite interesting. So that's why I will use TensorFlow as an example in these slides. So here you can see that we have TensorFlow GPU so the name and the pipeline unit that we will see shortly considers TensorFlow GPU as a pseudonym to TensorFlow package. If the environment has GPU. So let's take a look at prescription. All the examples that I will show onwards live in dot station prescriptions repository and that's the publicly available repository of prescriptions that the public instance of thought the cloud based resolver is using during the resolution. So here you can see the prescription. Now the pipeline unit has different type and that's pseudonym. Here you can see that the pipeline unit is registered if the given runtime environment has CUDA or CUDNN configured, not all, but both are configured in the runtime environment. And then we are matching packages such as TensorFlow Intel TensorFlow Intel CPU coming from pipi.org. And if the resolution process finds or encounters these packages, then the recommendation engine resolves or considers TensorFlow GPU as an alternative to these packages stated. So here you can see prescription, the version that is considered as an alternative is the same as the matched version of these packages. And then you can see again some log message and information provided to viewers. Then another example considers versions of CUDA. So that TensorFlow GPU is the version of TensorFlow GPU is correctly resolved. TensorFlow has support metrics when it comes to CUDA and CUDNN. It's sometimes it causes headaches to match these versions, but this cloud based resolver knows how to resolve TensorFlow GPU in specific versions so that the CUDA is utilized and the right TensorFlow GPU is used with the support it CUDA. The same applies to CUDNN that is basically another library used by TensorFlow. So here you can see CUDNN version. As you can see, these pipeline units can be very atomic pieces and can form complex resolution process that is respecting rules or prescriptions that are applied. Another example is fixing library overpinning issues. So in Python ecosystem, you can spot that libraries provide two broad range of supported version that are claimed to work. So here TensorFlow in version 2.1.0 and compatible overpinned version H5Py. So this prescription unit that is of type step makes sure that H5Py is resolved in a correct version that works together with TensorFlow. Let's move on to the next example and that is using specific Python package index that provides optimized wheel builds. Again, we have example with TensorFlow. So here you can see that when users use AVX2 enabled processors, then there is considered or sorry, prioritized builds that is available on the AICOE TensorFlow index. So in this case, TensorFlow in any version coming from this index will be scored positively and will take precedence in the resolution process over other builds of TensorFlow. You can see that this recommendation, that this pipeline unit is applied when people ask for performance, secure or stable software stack. Let's move on to example number five. In this example, we will see static source code analysis in action. So if you are a Python coder and you know this MKTemp function that is provided by temp file module, you probably know that this function is deprecated due to security vulnerability in race or is vulnerable to race conditions. It's also, this is also stated in the official Python documentation. So this prescription warns users about this fact when they use latest performance or stable or testing stacks. On the other hand, when people or users of TOT want to resolve secure software stacks, then the prescription errors and brings this error to users. So if this library function is called in sources, this prescription makes sure that users of TOT are notified about vulnerability in this function and the severity of the notification is based on recommendation type that users ask by the resolver. Example number six, checks runtime environment. So if people use environment where they have GPU, so GPU is not now and they do not have CUDA version properly set up. So the client too does not detect CUDA version. Then they are notified that there is no CUDA available and eventually other pipelining needs will make sure that, for example, TensorFlow GPU is not the result because there is no GPU to be utilized. That was example number six. And let's move on to our last example that is resolving Python packages considering RPM packages available in the runtime environment. In this example, we expect that Git, the RPM is present in the runtime environment. If it is not present, then this pipeline unit of type C filters out packages or any versions of package Git-Pyton. That means that the resolver will try to find another resolution path without Git-Pyton because Git-Pyton will not work on the host because of missing native dependency in the runtime environment. Then you can see this information is printed to logs. So if the resolution fails to find another resolution path, people will still see or users will still see that Git-Pyton is causing troubles when resolving software packages as Git is not present in the runtime environment. If you are interested in this prescription concept feel free to browse documentation and also feel free to browse the repository that holds these prescriptions for open source packages. We have bots that automatically update these prescriptions but also some user input will be available as if this database grows. That means that open source Python ecosystem will gain quality and the resolver can resolve better software considering knowledge about Python ecosystem. So this way we welcome you in contributing to the station prescriptions repository directly opening pull requests and implementing prescriptions if you're interested in this technology but feel free to also just report any issues that you spot in Python applications. The documentation for the declarative interface for the resolver is available also on totstation.ninja and I've already mentioned we have a homepage that is totstation.ninja which links to our YouTube channel so feel free to subscribe and you'll get also linked to our Twitter. We have totstation, Twitter handle without dash and here you can find updates and information about progress when it comes to development. Feel free to also reach out to us on this media and this way I would like to thank you and see you next time. Thank you. Yeah, thank you so much Frida for the talk. Folks, if you have any questions please feel free to put them in the chat and we have Frida here for another two, three minutes to answer them. Okay, so if you have questions I will be in the chat and this way I would like to thank you and hopefully you find tot valuable. Thank you. Right, thanks Frida.