 Good afternoon and welcome to the latest webinar in the BioExcel webinar series. My name is Adam Carter and I'm the host for today and the presenter for today, who will be telling us about all the new features in Gromax 2019, is Mark Abraham from KTH Royal Institute of Technology and also from the BioExcel project. So just to let you know before we go any further that this webinar is being recorded and it will be made available later through the BioExcel website. So if you need to point somebody else at it or to recap parts of it later on, you'll be able to find it there at bioexcel.eu. So before we make a start, before I hand it over to Mark, I'm going to give a very quick overview to remind people what we're doing here in BioExcel. We've now been running for over three years. We're a centre of excellence for computational biomolecular research and we're trying to promote excellence in three particular areas. So the first of all, first one is in software itself. So as a project, we are specifically supporting Gromax, HADIC, some aspects of CP2K in particular, the QMM interface for CP2K. And we're also working with other... Well, so they're the key codes that we are working on in the project. And one of our aims is to improve the performance efficiency and scalability of these key codes. Secondly, an important aspect of the project is usability. So as well as having fast and scalable codes, we want them to be easy to use, easy to integrate into workflow environments, for example. And to that end, we are working with software such as the CompSS and the Common Workflow Language as well to promote these things and to support usability in biomolecular workflows. And last but not least, we are also promoting consultancy and training in the areas of biomolecular research that we are working in with an aim to promote the best practices and to train end users so that they can make better use of the resources, the largest resources that they have available to them. So as we go through the talk today, I think it's going to be easiest probably to save up any questions that you have until the end. So there will be time to ask questions at the end. And we're going to use the question feature in GoToWebinar to ask the questions. If you do have a question, you can type it in at any time into the question area, but we'll start addressing these at the end of the webinar. So at that point, either if you have a microphone and you're able to speak, I'll invite you to open your microphone and ask your question directly to the speaker. Otherwise, I can pass your question on to Mark. So then today's presenter, Mark Abraham, is from KTH Royal Institutes for Technology and he is the development manager of Gromax, which as I'm sure that you are aware is one of the world's most widely used HPC applications. He has an ongoing role as a core developer in the team and within the team, he focuses on modernizing the code base to be sound platform for fast, flexible and free molecular simulations. So he's taking responsibility for the strategic plan, the release management and he coordinates the global team from his base at KTH in Stockholm. So you're hearing today from the person who does have the responsibility for the plans and the future of the Gromax code. So it's your opportunity to ask questions at the end if there's anything specifically that you would like to ask. Okay, so Mark then I'm now going to hand over to you. So if you would like to start sharing your slides then we will be able to start the main part of today's webinar. Thank you very much for that kind introduction, Adam. It's wonderful to be with you all here today and hopefully I can communicate with you some of the things that are going on in the Gromax code base such that we've been able to deliver in Gromax 2019 and I hope that you will find an exciting new tool in your arsenal for doing bi-molecular simulations. As many of you know, Gromax's strengths is in being able to simulate the kinds of biological macromolecules that are of great interest in many areas of both industrial and academic research. So we can see here on the screen a snapshot of one of the configurations being explored by a small startup here in Stockholm that are looking at how the process of skin drying out as it goes from the lower living layer to the outer dead layers takes place because they really like to be able to use molecular simulations to understand how to make these skin layers more permeable. That's quite difficult to do both in the laboratory and in the computer simulation so they've been using Gromax to understand how best they might suggest pharmaceutical companies maybe apply a cream to the skin to make it more permeable so that certain classes of drugs can be delivered that way. So that's just the very tiniest scratch on the surface of the kinds of things one can do with the sort of bi-molecular simulations that are possible in Gromax but we are continuing to make better all the time. A little bit of housekeeping to start with. Please do remember that all of our documentation is fully and freely available on the web where big believers in open documentation all of that is available immediately. If you would like to get a self-contained PDF this is something that we've been working on over the last year or two as part of the BioXL project so now all of our documentation including the all of the math from the physics documented in the former reference manual is now all available in the same place. You can download a large PDF but it's quite a few hundred pages so please don't print it out. Maybe print the chunk that you want and try to save a couple of trees. As a key part of that we have the release notes so as we change the code over the development cycle we keep track of those as we do and so those are made available to use so that as you are thinking about whether you want to adopt the newer versions of Gromax for your new scientific work you can see what has changed and what you'll be able to take advantage of in the new features. The new webinar about a year ago on the new capabilities in Gromax 2018 and I would certainly encourage you to check those out as well particularly if you're still using Gromax version that are even older still you might prefer to learn about all the things that happened then but I won't have time to talk over in any more detail now. We do have an annual release cycle associated with Gromax so we got Gromax 2019 out actually on New Year's Eve of 2018 so it's been out for a couple of months now this year and we've had time so far to make two patch releases so we started with 2019 and we've now made the 2019.1 and .2 patch releases we're planning to as people give us feedback and we find the issues associated with the code we'll release a patch version about every two months for the remainder of this year so we say that Gromax 2019 is inactive maintenance for this year and we still have also inactive maintenance last year's release so if we find some problem that would make it easy for someone to do science that they would prefer not to publish that might be wrong in some way we would make a critical fix to 2018 but we expect that these will be fewer and far between because that code base has been pretty stable expect a few rough edges around 2019 because we don't have 100% test coverage on everything but 2018 has been out for a good while now all the Gromax releases such as 2016 5050 or even four families are all now out of support they've been good software in their day and they will still work you will find on pretty much all of the computers you can have today but we can no longer afford to spend the time to continue maintaining those with bug fixes so we strongly encourage anybody starting a new scientific investigation to start with the 2019 version because that's the one that's currently under support and if you're the one person who discovers an issue then we'll be able to fix it there if unfortunately someone finds an issue in an older version if we can't replicate it in a newer version we won't be able to spend the time doing any fixes so we do encourage you all to get in touch with the latest version and please give us any feedback about how you about September this year we will kick off the process of making our 2020 release so you will see on the Gromax users and announce mailing lists that we've forked off the beta version Gromax 2020 that will have some of the tasty things that I'll hint at during this seminar will be available in those and over the final months of 2020 we'll be hardening that into a high quality release that you'll have in your hands in January 2020 so we've been keeping this annual release cadence for a couple of years now and we plan to keep doing it in the future as well because that makes it an attractive target for both users and developers to understand when the new capabilities will be available so that everybody can plan accordingly Gromax has been around as a software project for quite a few decades now we started getting our first citations back in 1995 although the software had been around a little while before that and we've had a number of quite successful publications over that time and as you can see we have been enthusiastically supported by the scientific community which we're very thankful we do ask that you continue to remember to cite Gromax in your published work in every time that you do use it have a look in your log files to check the paper that we recommend that you do cite if you're not sure which one to cite however you can cite the most recent one that you have to hand that will go great but we need to make sure that we keep being able to demonstrate to our funding sources and indeed to industry vendors who might want to make collaborative projects with the Gromax team that will help us be able to run faster and better hardware they need to be able to see that there's many thousands of people who like to use Gromax to do good science with it if you're publishing in something like Cell or PNAS or Nature or Science please do try to cite Gromax from the main text we realise they're often page limits and so on associated with that but if you're doing a large scientific project reported in one of these flagship journals we need to be able to see that Gromax back there so please try to cite Gromax in the main work not just in the supplementary because we've had such an active user base doing such high quality science with Gromax we've been able to attract numerous collaborative sources of funding some of which I will mention in passing as I talk about the kinds of functionality that they have been able to land in the Gromax 2019 release as well as things that have happened in the past that give context for those actions in which we'll be going in the future so we have as Adam has already mentioned the main BioXL project we have some Swedish projects that I'll talk more about some American projects several other projects in a few other countries that don't contribute too much to the Gromax core and we also have several active Vendor co-design projects that are all about making sure Gromax will work well on the upcoming access quality to the computer hardware that's going to be coming in the next few years so I won't need to say too much more about this particular slide because Adam's covered all that as well in the introduction but we are here in BioXL working on the core applications which is Gromax so we are doing several things in and around Gromax over the next couple of years supported by the BioXL project one of which we'll be integrating better support for QM and then based on the QM package CP2K so we expect to be able to do some revamping on the existing QM interface within Gromax and make sure that works very well with CP2K that will be coming in future years don't hold your breath for that any time soon but that is on the... so in Gromax 2019 we added and enhanced several features we got some feedback from the reproducible build project associated with the Debian project that they had some constructive criticism for us on how to make our build more reproducible we used to have a build that kept track of matter information about the computer on which Gromax was built which might be different from the one upon which it runs so that was occasionally useful not highly useful so we took good advice from them and removed that so this enables Gromax a Gromax build in future to be fully reproducible so that you can know that a particular scientific simulation used a particular version of Gromax that was built in a particular way which is our little contribution to the overall effort that science needs in order that all of us can reproduce each other's results in the future we had a contribution from a startup that I mentioned earlier who were trying to pull drug modules through skin layers in using Gromax as a modeling framework they wanted to add a feature to add pulling simulations that would be able to understand the geometric distribution of the group that they were pulling on which is important to get right with periodic boundary conditions might be going on and pull groups might be large or dispersed or moving a lot over the life of the simulation there's a minor feature there that allows that center of mass to bookkeeping to be done from the previous step which is much more stable and much more capable than they started earlier in the simulation so that's a nice little improvement for any of you doing large pulling simulations one of our analysis tools GMX cluster now produces B2B files that are a bit easier to use so this was again work supported by our Excel that helps make Gromax easier to use our normal analysis tool GMX NMI now produces thermochemical results just like quantum packages do so they can estimate things like heat capacities and so forth based on harmonic approximations so that's a nice little feature if you're doing that sort of work and we have a new tool that we forked out of the old GMX dump there was a dormant feature there that would write a little method section for your publications the GMX report methods that would look at the TPR that you used and write some so do check that out and give us some feedback about how well you find it works we'll be very interested in seeing how useful that is we also had to remove a few features from Gromax because we added them a few years ago when we thought they were a good idea but sometimes ideas that were good then aren't any good anymore so NVIDIA had a while ago a feature called NVML that would allow Tesla grade GPUs to be able to have their clock speeds changed that feature still around but now feedback from users and compute centers wasn't that very few people were able to take advantage of it so we decided we would remove that and simplify the code similarly NVIDIA's GPUs have changed a lot since the early days and the current that we had to support the so called Fermi class of GPUs which are about 8-9 years old by now we're getting a bit outdated making our code more complicated than it needed to be so that we can continue to implement high quality things as correctly as possible we removed that one as well those of you who are familiar with multi-simulations perhaps from doing repure exchange might be aware that we had two ways to run multi-simulation there was a minus multi and a minus multi da we removed the minus multi one in favor of the multi da one because that's a little bit easier for people to use and it's much easier for us as developers to maintain and test if there's only one way of doing one thing so that's something we announced and deprecated a while ago and we've removed one of the implementations now so we appreciate that that's going to be a little bit destabilizing for some people but it's also going to be an improvement in some ways there were several features that simply couldn't be implemented well with minus multi so multi da is the path to the future so please update your scripts for that we also decided to control that on our strengths and remove the implicit dilation feature that we introduced in Grimax 4.5 unfortunately no one had the resources to maintain that for some years so we bit the bullet and removed it there if you are interested in doing implicit dilation with calculations we strongly recommend the implementations in the amber package of our numbers and if you want to use those sorts of models that absolutely can use those they're very high quality we also removed support for the BlueJeans family of supercomputers they've pretty much all been taken out of service now so we got our code a bit simplified in that direction because time is moving on because we need to be able to take advantage of the new features that have been out that many people have available already in their clusters we now require CUDA version 9 that's about 2017 era CUDA generation correctly might have that wrong by year or so but most of you should have access to clusters that have at least CUDA 9 installed so you will need that for the Grimax 2019 releases and moving forward we'll probably increase that roughly every year so that we can continue to take advantage of the capabilities that are in the newest hardware because particularly the GPU hardware is changing at a frightening pace we as developers need to keep up with that while not requiring you to install new stuff absolutely every time you need to upgrade Grimax so we have to strike a balance we do have a Windows support in Grimax which we're very happy about because it helps make sure that we keep our primary workhorse platforms Linux more portable than they would otherwise be however we're only able to support the very latest versions of the Microsoft Compilers so 2017 is required in 2019 and with that if you are trying to use GPUs with Grimax you'll need CUDA 10 for that slightly higher requirements there if you're using our support for OpenCL GPUs which is particularly useful at the moment on AMD GPUs you'll need a run time support if you've got a serious implementation of OpenCL these days to support that so we decided that it would be good for the world for us to have a similar and more correct code we did also announce in Grimax 2019 some things that are going to be changing in future GMX-MD run has been our simulation workhorse that has done an awful lot of things we've involved calculating some forces and changing some coordinates but not all of them were really MD in the sense of being molecular dynamics we also had things like membrane and braiding preparation tools and energy minimization and reruns that would take the coordinates instead of being produced by the forces would read them in from a file we also had options on the MD run that were useful for benchmarking but not really interesting for anyone doing a production simulation documents, tests everything around GMX-MD run so we're going to be simplifying the user interface quite a bit by moving those things around and hopefully that will make it easier for us to make sure we get it as well those of you who've been using Grimax in teaching have also probably shame-facedly had to observe at some time that Grimax's integrator MDP option also allows you to choose different kinds of energy minimization functionality that's a bit silly because energy minimization is an integrator in any kind of sense so we'll be changing that there as well we've been doing a lot of work behind the scenes that hasn't landed in a released version of Grimax yet that's going to break up a lot of the coordinate frame transformation capabilities that are in to all of our GMX church convenicons and friends like them we've been hearing feedback from users for quite a long time that it's a bit frustrating to have to run an MD simulation and then how to run TRJ cones to get the bits of your simulation that you care about making sure that they're all the same simulation cells and the same equivalent periodic boundary representation we've found out ways to implement those as pre-filters that we'll be able to make available as one path analysis tools in future versions of Grimax things that you used to have to use Churchcon perhaps multiple passes of Churchcon for so we're just letting you know that GMX Churchcon is going to be the path for the future but there's no need for you to change anything about how you work with that now also there has been a feature for the last few versions of Grimax where you could tell GMX MD run please ignore the number of steps in the TPR file I want you to do this many steps that too was a bit hard to understand and document and it was really only useful for people doing benchmarking studies on Grimax that only a few of you out there were using that as part of your production workflows but if you are please be advised that we'll be going away in Grimax 2020 and hopefully we'll be replacing any need for that benchmarking with a new tool GMX benchmark moving on now to some of the work that we've landed in Grimax 2019 so that you can hear about some of our new capabilities the Swedish EScience Research Center supports one of its core projects CESI which is the Cirx Exascale Simulation software initiative and they support two large Exascale software packages Nex 5000 and Grimax running on the latest computer hardware in and around the HPC scene so for Grimax they fund the developers that do most of our work on the GPU and CPU accelerated versions of Grimax so we're going to look a little bit about the functionality that came in there those of you who have been using Grimax for a little while now understand that we have a state-of-the-art 3D domain decomposition functionality within Grimax that is able to take any arbitrary kind of simulation cell so here for example one might have chosen a rhombic dodecahedron to model as an approximately spherical but periodically tileable unit around an approximately spherical protein we can take this rhombic dodecahedron for example and express that as a triclinic unit cell which is much easier to do with the mathematics on about what happens when a particle goes out one side of the periodic boundaries and comes in the other side we've had for many years in Grimax the ability to take those triclinic cells and break them up distribute them over multiple different nodes of the supercomputer and make sure that the load gets balanced by having the domain boundaries stagger backwards and forwards so this was reasonably effective for a long period of time we were very happy with that high quality implementation of how that worked however we discovered that some of the design choices we made when we implemented it should be re-thought so we have a new feature called update groups in the past when we did domain decomposition we just did like these blue lines you can see we just did a vertical chop or a horizontal chop straight through the system and we didn't worry too much about whether we were crossing any particular chemical bonds or any molecule boundaries as we were doing that we just said okay we're going to do a straight geometry decomposition and this worked very well for the force calculation it pretty much doesn't matter where bonds are however it was a pretty bad idea when it came to the update phase we've discovered on highly parallel resources that we could be spending almost as much time in the update and constraints phase as we did calculating the forces even though the orders of magnitude less computational work in terms of floating point operations that one has to do during these phases the reason for that is that if you did chop I'd say a carbon-hydrogen bond which you can see here and had constraints that needed to be satisfied if you gave one know the responsibility for updating the hydrogen and another know for updating the responsibility for updating the coordinates of the carbon then they have to coordinate multiple times during the update step to make sure that the constraint on that bond length was satisfied and that particularly high in scaling which is such as we need to focus on as part of Erics's scale funding wasn't very effective so instead what we have available in Gromax 2019 already is the ability for our domain boundaries to rather than being a naive slice straight through to be a little bit staggered in fact what we do is compose the domain out of what we call update groups so this carbon and this hydrogen if the constraint option chosen is that all bonds between heavy atoms and hydrogens are to be constrained they form a constraint group because the other atoms bonded to that carbon and heavy atoms such as carbon and so in this case we are able to describe this domain out of the union of the set of these update groups and they've been chosen so that when we're doing the constraint phase we don't need to do any communication so those of you running large scale parallel simulations are likely to see a tidy performance benefit here simply from the fact that we've avoided a whole bunch of communication that was otherwise painful for you so you ought to see either higher performance or higher scaling on higher resources we hope to be able to leverage that in future with other developments that are going to be coming in Gromax but also we have a number of active co-design projects with vendors who want to produce hardware that's effective for large scale simulations on civic computing resources in the long term so we have active collaborations with quite a few of these companies including NVIDIA, ARM, Intel and AMD some of those are larger scale and one of longer duration than others some have come and gone but we're working actively on the hardware that we see coming out over the next few years to make sure that on day one there will be a released version of Gromax that can do a good job of running on it and as we see that hardware has stronger impact on the HPC sphere we'll be continuing to do our best to make sure that they run optimally on those taking advantage of the features of the hardware that involves porting Gromax to use GPU acceleration libraries many of you will be aware that we've had support for the CUDA acceleration library from NVIDIA for quite some years they do an excellent job of producing cutting-edge hardware and the library and runtime and drivers that makes those work we also have support in Gromax for the vendor neutral language OpenCL which you can use to run Gromax right away also on NVIDIA devices and on AMD and in Gromax 2019 one of the new features we added support was for the ability to do that also with Intel's integrated GPUs on laptops so all of that technology was actually brought to us by StreamHPC which is a Dutch company that contributed the OpenCL port of Gromax OpenCL and vendor neutral computational threat with them because that has added value to the future collaborations we've been able to have with AMD and Intel so thanks very much Vincent and the team at StreamHPC your legacy is living on and being great positive for us with background information on the algorithm that we currently have implemented in Gromax this is not new work here we landed this in Gromax in 2016 already we have the need in any AMD package to do something like a pair search so we maintain a list of particles that are likely to be within the interaction radiuses for the short-range interactions we maintain a pair list and update those periodically during the simulation and in the actual AMD steps we have a false calculation that involves computing on bonded interactions non-bonded interactions sometimes those are called short-range interactions and along the range of PME interactions once you've got all the forces now we need to integrate the equations of motion turn our forces into a velocity update then into a position update then perhaps constrain those updates so that we stay on the list with the constraint description then we iterate back over this AMD step every so often the enableress has expired we need to build another one we found in earlier versions of Gromax because it added yet another parameter that we would want to tune the lifetime and their balances of now the excess flops we do versus some relatively poorly paralyzed bit of the code so a few years ago this project did some great work to move us to a dual pair list where we have an inner list that we update pretty frequently and an outer list that we update very infrequently and this has been great because it'll be able to take a parameter out of the user's hands and deliver higher performance all for free unfortunately this meant that our code got quite complex so we'll have a look now at how things work in the guts of Gromax on the next slide so if there's any impressionable children out there please make sure to cover their eyes because things make a little bit gory right about now so the key advantage we got from this single pair list was that we were able to take more advantage of the waiting periods where we didn't have much particularly for the GPU to do during the integration constraints phase we do that back then on the CPUs only and we still have that attribute in the Gromax code still today what we were able to do however was to run this rolling pruning calculation so that we can update our inner list in a rolling and staggered fashion over the duration of the outer list and thereby have an efficient inner list and also make use of this dead compute time so that was a nice innovation that made things work well however in the overall scheme of things we're getting pretty complicated these days because we have to maintain naval lists of particles that are shared with neighboring domains that we're going to have to communicate over in PI we've got other short range lists for particles that we're only going to be able to only have to communicate with in a local sense we won't have to send anywhere else so we do have different priorities for those kinds of work that we need to do in the work and at bonded work the internal details of Gromax are very complex so this is why we are vigilant to make sure that when features and things like GPU versions go out of date that we remove support for those so that we keep this as simple and maintainable and correct as we possibly can one of the things we also did for for Gromax 2016 and expanded in Gromax 2018 was to the PME calculation to be able to run also on GPUs and so in 2016 that was available on CUDA GPUs and in 2018 that was also available on OpenCL supporting GPUs so that's quite fantastic because it means that we're able to take advantage of the extremely cost effective particularly consumer grade GPU hardware that's out there and can spend less money on extremely powerful multi-core CPUs. We still needed those in Gromax 2018 for computing the bonded forces and for the integration and constraints phase but it was less costy than it ever was before. In 2019 however we added more GPU support for that recognising that the most cost effective way to get flops on modern computers is using the GPUs we added further support to free energy perturbation simulation such as you might use in alchemical drug design any FEP calculations that are doing PME that aren't disturbing the chargers can now take advantage of the fact that PME can run on GPUs. We had that turned off in early versions of this functionality just because we hadn't gotten around to testing that would work but it did work very well so we turned that on in Gromax 2019. Unfortunately you can't use PME on GPUs if the chargers are being perturbed because we haven't afforded those particular kernels yet but that may come in a future version. Of course we added support for the latest generation of NVIDIA's GPUs and made sure that thing also worked with the latest GPUs from AMD and Intel as well. We made sure that the PME long range interactions as I mentioned run with OpenCL on also sorry that these long range interactions also run on OpenCL devices particularly from AMD so you can get access to those by making sure that your own command says hey I want the PME calculations to run on the GPU. Thanks to our co-designed projects with Intel particularly Roland Schultz we now have support for running using OpenCL on the Intel integrated GPUs that many of you will find on your current generation laptops. Any laptop that's come out the last couple of years probably has an integrated GPU that's able to take advantage of our OpenCL port and if you're using your laptop for example for little energy minimization or collaboration workflows you might find 25-50% speed up. One thing I didn't realize that I learned in a talk recently that if you look at the amount of silicon that there is on the CPU in typical laptops the GPU that we're now able to take advantage of is actually bigger in terms of number of transistors and CPU die area than the CPU so it makes quite a bit of sense that this is a useful speed up for Gromax so you can make your local builds of Gromax on your laptop using this combination of flags down here unfortunately we didn't have enough time last year to make a cluster size selection automatic so unfortunately it's a build time option but of course you won't be using that build on the other kind of hardware so we hope that's not too big a deal for you but you won't be able to take advantage of the Intel GPU unless you do that. We do have other ongoing projects that are looking to make Gromax work on Intel's upcoming line of discrete GPUs that's probably going to be shipped under the Intel XE grant so Intel have changed away from their strategy of landing at night middle families of processors towards more like a discrete GPU so we're going to be working very hard with Intel to make sure that any products they're delivering there in the near future will be able to run Gromax so watch this space. Meanwhile we had a very effective co-design project with Nvidia whose engineers were able to move a large number of the bonded interactions that are in common use in molecular force fields able to be run on GPUs. We landed that also in Gromax 2019 such that this last diagram that you just saw has even less work that it needed to run on a CPU so we're continuing to work with Nvidia's GPU development team to also pour our integration and constraints along with the CESI project supporting some development efforts there so that we're hopeful in Gromax 2020 to be able to have certain kinds of simulations able to run purely on the GPU naturally there will be plenty of calculations that have yet been imported exactly what we're going to be able to do we don't know yet because we've still got quite a few months of development work in front of us but in particular the work we're doing in collaboration with Nvidia is making sure that we're able to take best advantage particularly of the capabilities of the very high end hardware but we'll also be making sure that things work on the consumer very hardware that we know many of you have. Finally we have another active project that's funded in America from the National Institutes of Health that's been running for a few years working in two major directions one of which is a full-scale real-life of Gromax's integration functionality because we've had capability to run both LeapFrog and Vosly Volay families and integrators for quite some years however there's an awful lot of exciting things we would like to be able to try doing including having things like Monte Carlo Barastats and perhaps multiple time-stepping integrators which would leverage the update groups feature that we've already landed in Gromax 2019 so we hope to see some of these features landing in Gromax 2020 but to do that our colleagues at the NIH have been working very hard on a complete rewrite of the integrator loops so we are going to be landing some of that code shortly. Similarly other colleagues supported by the NIH have been hard at work you may have heard already in this particular series about the Python API that we have in development for Gromax we did land an early version of that in Gromax 2019 with some extremely limited functionality that you can hear about in last year's GMX API webinar in this series from BioXL and you can also read about it at that paper so Gromax 2019 is capable of doing all the work that they are describing in those pieces of work there we're landing expanded versions of that as a first class thing that we supported in Gromax 2020 that's looking to be an exciting piece of infrastructure for the future that's also how we hope to leverage the re-written functionality that you formerly people used things like Terrage Covenant Con for the automatic pre-filtering technology that BioXL has been developing there will be made available through the GMX API technology so that we can get away from these multi-pass command load workflows and instead do everything in single-pass Python scripts so that should be an exciting usability improvement now in the longer term we have a very large number of things that are being worked on both in the core team here in Stockholm associated with the core team perhaps with a little bit of funding here and other people supported elsewhere and indeed some things that are fully remote I won't go through each of these points here I've talked about some of them already but there are lots of things where in particular hoping to have container-based distributions of GMX in the future which will hopefully be easier for distribution in things like BioConda and using with singularity on supercomputers so that you have as easy as possible a time installing and running GMX and we have some funded effort also that's kicking off shortly to make a high quality C++ library around our high performance and on-bonded both enable us to start grappling with some of the performance complexity we saw earlier and hopefully make that available to other people doing software development that needs the kind of functionality that we've already done a good job of making fast and highly portable in GMX we'll be able to do that without needing to spend a lot of time rewriting their code for GPUs and CPUs and what have you so there are lots of exciting things coming some of which will land in GMX 2020 which have longer timeframes but we hope that we'll be doing the things that you want to see in your simulation packages in the future so as you might expect there's a tremendous number of people that one could acknowledge over the nearly 30 years of GMX development many of those people have moved on to greener pastures and I haven't listed all of their names here but we have here numbers of people who over the 2019 time frame contributed to the functionality that we see in 2019 so they all did a wonderful job and we're very thankful to the efforts that they've been able to put in and also to the various funding agencies including the European Commission and the USNIH and the Swedish methods rather have supported a lot of these people in their work naturally there's several people there also from particular vendors particularly including NVIDIA and Intel thank you very much for your contribution so that brings me to the end of the material that I prepared in the presentation as Adam said earlier I'll be most happy to take any of your questions I hope you've got lots of thoughts for me thank you very much indeed Mark that was great so yes I can see that we've already got some questions coming in so yes a reminder to anyone who wants to ask a question you can type it into the question area in go-to meeting which you should find on the right hand side of your screen and then I can either invite you to ask your question directly or you can read out your question so I see the first question from Sergio Pantano Sergio do you if you have a microphone feel free to open it now and ask your question let's see maybe I have to open it so let me just try so we've got quite a lot of people in the room to find you in the list okay so Sergio do you would you like to ask your question okay I'm not hearing Sergio so the question is simply are you planning to implement double time step for multi-scale simulations does that make sense to Mark yeah so there are plans to implement a multiple time stepping functionality that's going to leverage this update groups technique that I talked about in live in them no I can pass it back to you now okay you should be able to as part of the integrated refactoring that the NIH team is doing Pascal Metz and Michael Schratz have been working on the functionality that we will need for a multiple time stepping framework so that kind of functionality would be enable us to run the long range PME contribution much less often perhaps only every 40 seconds while we continue to run the short range and bonded work at a higher frequency which would offer the kinds of performance that we use to get using the virtual sites plus constraints framework that we have in Gromax that we now think having tried it out for a few years is not a viable way to get high performance in simulations as these multiple time stepping approaches will be so yeah we are actively working on that we're very very fortunate that the NIH project has already been doing the necessary work to give us the composable building blocks so that we can have some of our force calculations happening both in a rigorous way using a proper Toronto style decomposition and also okay thank you Sergio I hope that answers your question if it doesn't then feel free to ask a follow-up so next we have a question from Matthias McCarder so I'm going to try to open your microphone Matthias would you like to ask you oh it was a general okay Matthias didn't have a microphone he was just I think the main point of his point was just to thank you Mark for the update and to tell you that it was very useful actually I don't see any other specific question there yes hello there is actually I'm just noticing there is something in the question that Matthias has asked which is to say he's a fan of Gromax however sometimes it is confusing do you want to read the rest of it yeah okay thanks thanks Arno so the point for the Dan that I hadn't noticed was so sometimes it's confusing to choose amongst active branch codes for example Max 2018 versus 2019 series the same happens with releases there were seven versions in a year so he's looking for practical recommendations on how to choose the most stable version for a scientific production and then there's the follow-up is it possible to calculate short-range and long-range interactions PME on multiple GPUs so two questions and one there so on the first question when we make a release I'll say about 2019 branch the functionality has been stable for a year so in terms of which is the most recent version well that's the one with the most recent year the most stable version of so there's no difference in stability between any of the versions that we do release there were changes between 2018 and 2019 principle they could have introduced a bug that we haven't caught yet we can't prove that there isn't a bug that's because cheering in complete question I'm sorry there's no particular advantage in stability of choosing some old version particularly if it's out of support that's a way of guaranteeing you're choosing a piece of software that has bugs in it because we don't actively fixing the bugs in those old approaches anymore so we have simulations using a recently released version of Gromax both 2019 and 2018 are under active support if there are bugs we find we will fix them we will make changes in those branches with a view towards keeping things working that already working the only things we do in these branches are fix bugs we don't add new capabilities so from that point of view things are as stable as they are between all these different versions the number of versions that get released in the year doesn't say anything about the stability of those codes all that means is that we found issues that were worth fixing and we got those out there this is no different from any other kind of software out there they release versions that fix stuff for example amber simulation packages release patches to their released version periodically as they identify issues they have the same kind of challenges as we do they have a slightly different way of doing it but it amounts to the same thing there are minor changes made to the code so always use the latest patch release of whichever branch that you're intending to do and keep looking at the release notes that we issue with each of those patch releases to see whether there have been any issues fixed the second part of your question referred to can we do PME on multiple devices there is a branch of Gromax that's under preparation by NVIDIA particularly that would like to take advantage of particularly the capabilities of the Tesla grade hardware particularly leveraging NV Link the high speed networks that are available between GPUs so they are hopeful of being able to integrate some code for that into Gromax 2020 a long road behind that so no promises on that at this time there certainly are people working on it it's a huge technical challenge however fortunately projects like BioXLR are helping us work on the underlying library infrastructure to help refactor things like so it becomes reasonable and this new non-bonded library project will also be helping out for those kinds of things so yeah we'd very much like it to happen in the future it's likely only to happen in the context of being able to do data transfers directly between GPUs so the only technology that's currently on the market that can do that is in NVIDIA specific but obviously will have area to the ground to make sure that as new technologies come out that are effective for that we'll try to support those as well okay thank you Mark so the next question is from Anna Kett would you like to ask your question directly okay so don't have a microphone so the question then was it was noted that simulation performance drops down more than 50% when using virtual sites with Gromax 2019 do you have any thoughts on this Mark there's a lot of scope in virtual sites I can't comment on any particular kind of general phenomenon so there's a follow on statement I don't know if they're connected there's a follow on a question that says free energy simulations so I don't know whether they're intended to be linked or not right so certainly free energy calculations didn't get any improvements in Gromax 2019 they didn't get any pessimizations either if you think you have a simulation that ran at all faster in 2018 or earlier versions in 2019 we would be all ears to hear about that and to work on fixing it if you mean that there's performance degradation between running the same simulation without free energy perturbation or with it yes that's a phenomenon that we're well aware of and working towards fixing the reality is that the kernels that we use for doing the kinds of free energy perturbation calculations haven't had anywhere near as much of the high quality software engineering as the mainstream simulations have had that wasn't a problem when people mostly did FEP on a single iron or a methane in water or a 12 or 20 atom drug molecule but as people are working towards larger and larger workflows the need for those calculations to run faster is becoming more and more critical we do have plans to fix that for Gromax 2020 one thing that is worth mentioning that I actually should have mentioned in the main text of the presentation that I didn't is that if you are running on GPU accelerated simulations and you are not perturbing the charges then you can make sure that you get as many parts of the calculation as possible off the CPU is to leave those free for these interaction kernels to run so if you make sure that you put your number of calculations both parts of your PME and the bonded interactions on the GPU you will maximally free up the CPU for running the relatively quick calculations you get better mileage for that process if you are running on Tesla grade GPUs from NVIDIA then you will from running on consumer grade GPUs either from NVIDIA or AMD but those are some tips and tricks that I am thinking about in those cases I don't think virtual sites themselves are actually relevant to the question okay thank you Mark so Aniket if you do have any other follow up just type it in but hopefully that answers your question so the next question which I am just about to find here is from Zac asks are there any plans to incorporate or improve modeling of Druda force fields into the main GROMACS release there certainly has been some work done particularly by Justin Lenkul and his collaborators a few years ago so there is a fork of GROMACS 5.1 I think it was that got down a couple of years ago that does have reasonably useful version of GROMACS that does have support for that there haven't been any moves to incorporate that into mainstream versions of GROMACS that would be a considerable piece of work because we would need to modify a lot of the performance core force calculations still it is a very exciting piece of work and it is a way to improve modeling accuracy and take advantage of the ever increasing flood density in the part where there is nothing in the pipeline that would facilitate that but it is something that we would be interested to talk with people who have enough bare software to develop the cycle to prioritize work on that a little bit of a chicken and the egg problem there you need a high quality implementation in order for people to show that there are important problems that need these kinds of treatments I understand that particularly simulations in DNA and RNA are particularly improved by these kinds of polarizable force fields so we will be keeping our eye on that and when the opportunity is right to make those things work better but for now I do encourage people to check out Justin's report and give him some feedback on how you find those work okay thank you next question is from Amin Sagar Amin I am opening your microphone if you are able to ask directly no okay so in that case next question to summarize it says are there any benchmarks for different GPUs and CPUs like the ones available for AMBA to help to decide what kind of hardware to buy very much there are they have even been computed as part of there is a pre print out at the moment that updates how one should go about using hardware to run for GROMAC title of the paper is more bang for your buck improved use of GPU nodes for GROMAC 2018 and the conclusions from that are pretty much always applicable to GROMAC 2019 with the caveat that in some cases if you are using Tesla grade hardware you will be able to get benefit from the ability of GROMAC 2019 to also offload some bonded interactions in the same way so that is currently under review but that was worked on by some of the core developers in FESI and BioXL so that is a very high quality piece of work that provides guidance on what sort of performance you should be able to see and which sorts of hardware you might want to consider buying or renting to run GROMAC form so yeah very very strong thumbs up for the work they have done there that's great thank you Mark so I think we have time for one or two last questions Zach had an additional one as well he asks considering the wide use of Plumed with GMX is testing taking place to ensure they are compatible and which version of Plumed should we use with GMX 2019 I know I have seen comments on Plumed that they are planning to support a version of GMX 2019 that's up to them when they make their next release I'm not quite sure what their release timeline is but we don't collaborate closely with or deliberately test GMX with Plumed one of the challenges there is that of course they intend Plumed to be a project that works with multiple MD packages so while we're very glad that they are able to make downstream use of GMX they want to keep a little bit agnostic with respect to simulation engines so they haven't upstreamed any of the code that might be useful to incorporate within GMX at some point I understand that may have happened for LAMP so we could consider doing that in future also for GMX so if you're a Plumed developer and you want to help make that happen then please do get in touch with us but no we don't don't specifically test so yeah I would use check out the latest version of Plumed and hopefully they support GMX 2019 I know that they support some of the GMX 2018 versions but as always use the versions of GMX that the Plumed developers say they have tested because they're the ones who understand how to test their software well nothing much has changed in GMX that affects how they do use GMX so I expect probably a more released version of Plumed will work with 2019 but follow their documentation please okay thank you for that advice Mark and I think we are now we're coming up to the top of the hour and there are no further questions in the queue so I think and I'd just like to take the opportunity to thank you very much again for a very interesting talk and to remind everyone that they can keep in touch with us through BioXcel website and click to sign up for the newsletter it's the best way to keep in touch with what we're doing and there's also a page bioxcel.eu slash webinars where you can always keep up to date with the webinars that are upcoming from BioXcel so thank you all for coming along today and thank you Mark for your talk today and we hope to see you all again at a BioXcel webinar or event very soon bye