 Thanks everybody for for joining me today. This is a great venue. This is like I mean normally we're in a hotel like conference room or a ballroom or something and it's like the most stale environment ever like this is this is a really nice change of pace and thanks again to Johnny for putting this all together and PJ and in the people that seem to be doing the most amount of work which is Johnny's family so thanks to them too and I was telling some of these other guys that like literally Girl Scouts can come to my house twice a day and I'm gonna buy cookies like every single time so like thanks for having the cookies here it actually was really I mean those little shortbread cookies are just I mean they're the bomb so all right yeah so thanks everybody this is a presentation I'm calling a deep dive into Hex so Hex is a tool that I bet everyone or almost everyone in this room has used but probably not too many of you have given a whole lot of thought to it my alternate title was wait it's more than mixed-depth get and laugh come on man all right tough crowd but oh yeah get that in post okay so yeah for most of us like mixed-depth get is about the closest we get to Hex and we we take advantage of the fact that Hex is just gonna do its thing and it's always gonna work and I think that's one of the the parts of Hex as a package manager that that excites me is the fact that it really just does its thing and it doesn't really make a nuisance of itself so introduce myself my name is Todd Resodec I'm a senior software engineer at Weedmaps we're a primarily elixir operation at this point although we still have some ruby stuff and and we are currently hiring so if anybody's interested let one of us know I think there's six of us here today so we're looking for people with experience in elixir and some people that maybe don't have a lot of production experience or any but have an interest in it so just let us know talk to us and you know we'll give you some more information online you can find me at super simple or some derivative thereof like on Twitter with no vowels so and you know by all means like find me on online or find me like after the conference or after the talk I like to meet new people and you know this is a good crowd of people so so yeah let's let's be friends all right so what is Hex so let's define it first so Hex is the package manager for the Erlang ecosystem so it's analogous to what Ruby gems is for Ruby or crates is for Rust or npm for node all right where did it come from it came from Sweden this guy Eric Meadows Johnson is the is the author of it and it was released in December of 2013 so you know a little bit after Jose released the first version of elixir so how how would we compare it to some of those other package managers so one way would be the amount of libraries that are available so Hex has right now about 6,000 packages available Ruby gems has about 9,200 and let's see crates is about 14,000 which if anybody's interested in crates come find me after this because I think that's actually a really great piece of software too and npm yeah I like I feel like I should comment on that number but like everybody sees that and realizes how ridiculous that is so the only thing I'll say about that is suffice it to say that quantity and quality aren't always the same thing sometimes they're mutually exclusive in fact so I'm sure there are 6,000 really good npm packages so and just to give you some idea of like where these like where the growth is happening so since hex hit the 5,000 mark so in the in the time that hex gained 1,000 packages Ruby gems gained about 400 packages and so I've been tracking this for about the last year and definitely seen what looks like and people may be moving over to you know evidence of people moving over to hex or maybe just maybe just 9,200 is like the magic number of valuable libraries in Ruby gems and and hex is still catching up but so why why would you want to know more about hex basically so some ideas would be to work faster to prevent issues from happening or to debug a little bit easier and I'll explain on those a little bit later so here's the anatomy of a hex package and you'll notice it looks a lot like any other mixed project the one thing I would say is is maybe a little bit different is the license markdown file and you won't find that in every hex package but just keep in mind that like one of the the rules of hex is that if you release your package under a specific open-source license that you have to include this file in there with the complete text of that license so just keep that in mind if anybody's like thinking about making a hex package everybody can read that right all right just kidding so the this is the mix file for a common hex package so you know it's gonna look again like most other mixed projects I just want to point out a couple things so the version versioning is required to be semantic which you know hopefully you're doing that anyway but just keep that in mind that hex requires semantic versioning the description which is in there this is a really important piece of data for a hex package because this is one of the two fields that's actually searchable so if somebody's looking for a package that does a certain thing or has a certain name this is one of the the two places where that text can exist and it's gonna come up in search results it's also what everybody's gonna see on hex PM so when they go there and they try to figure out is this the package that I need to use this description is really what's you know make it or break it on that the depths which this is kind of like inception because this is a package requiring packages and this isn't hex specific but I think there's some things in here that relate to hex that a lot of people especially newer people don't know and that's the fact that you can load these dependencies from a bunch of different sources so probably the most commonly it is to load it right from hex but if you can see like on the second one that's loading in from github so that's instead of going out to hex PM and retrieving that package it's going right to a github URL for that or you can include a path and so that just points to your local system so the reasons why you'd want to do that would be a it's a library that you don't think is very useful to anybody else or it contains proprietary information that you don't want to open source and put out into the world or be maybe you're developing a hex package and you want to test it locally as a dependency so you don't have to push it up to hex as an you know unfinished or untested package you can load it in as under the path flag and and test it from your local the package so this package section of the the mixed file is for hex and the important thing to look at here is the name so that's the other form of data that's searchable on hex and that's essentially what your package is called so when people say you know I downloaded the cowboy package if you look in there the package name on that is literally just cowboy and sorry the last one which is you may see or you may not see sometimes is the organization and is anybody here have a hex organization set up like raise your hand or we got we got one I know we we maps we do so a couple so organizations are private repositories so it's like hex PM that's sheltered from the rest of the world so if you're pulling in your package or you're publishing your package to a private repo this organization will set and that's going to tell it what organization it belongs to all right package naming so I want to read this verbatim there's only one rule in hex about naming your packages and it's this avoid using offensive or harassing package names nicknames or other identifiers that might detract from a friendly safe and welcoming environment for all and I want to read that verbatim just because I think it is really important and you can probably think of other ecosystems where the maybe the main contributors or they're just the community in general is not very inclusive and you know it turns into this flame war 4chan sort of a thing and I think elixir Jose and Eric in this case have done a really have been really proactive about making sure that everything that happens in our community is made to be really friendly and you know if you've ever looked at the pull request that people make on elixir on the elixir lane repo you know Jose's always putting unicorn emojis and hearts and stuff in there it's never like hey you're a beginner or noob like this is totally wrong it's always meant to be really encouraging and and so again like this follows along that same theme and again I just want to reiterate it because I think it's really important that we keep our community as inclusive as possible Jeff thank you I mean it really it is it can't be understated the fact that like we might have this great programming language but if the community devolves you know we're gonna lose smart people new people and it's I mean it's eventually he's just gonna atrophy so so yeah now there's only one hard rule for naming but there are conventions and so one of them is when you're adding functionality to an existing package use that package's name in your package name so for instance if you're building an authorization tool that goes on top of plug you'd want to name your your package something like plug off so that people know that like this is an addition to plug you know just so that's very clear so and in this case you'd want to avoid namespace conflicts so if you're if you are making plug off for instance your modules shouldn't be called plug dot off because that's encroaching into plugs namespace so call it plug off camel case that should be the namespace that that you use for your package and this this one I think Johnny disagrees with this one but I think it's kind of the convention is when when porting a library from another ecosystem it's typical that you're gonna add ex to the name so for instance like the faker gem is I think it's called ex faker but you'll commonly see like if it's a sorry hex package so you commonly look for like a ruby gem that you know of by name it's gonna have that same name underscore ex or ex underscore that name and that's just a kind of a common indicator that this is functionality that you're used to in this other ecosystem but this is version is an elixir and names are first come first serve so there's no preference given to trademark holders on names so for instance if you if you decide hey look Spotify doesn't have an API wrapper available on hex I'm gonna write one you're totally able to call your package Spotify you don't have to change the name or you know spot if the cater whatever there's no mandate from the hex community that that you give deference to the trademark holder having said that name squatting isn't allowed in hex so we for instance if take the Spotify example if you say like oh I'm gonna build that Spotify API like tomorrow so I'm just gonna put up some trivial package now to sort of stake my claim on the name you know how that goes like tomorrow turns into next week and then that's next month and then it's six months from now and you still haven't really built the Spotify package so in those cases if there's anything trivial if there's a package that's deemed to be trivial the administrators of hex will just take it down alright so what hex commands should you know and this is the first section which I call works on my machine and I this is what I always tell our QA team and they which they love so the first one is mix hex info and most of these I'm gonna go over a mix tasks so I might just leave off the mix but hex info just type that into your terminal it's gonna tell you what version of hex you have installed what version of elixir which version of OTP and then it's gonna tell you what it was built with and then also at the bottom it's gonna tell you if there's a newer version available so why would you want to know that let's say if you're having a problem but your co-worker isn't having that problem with the dependency it may be that you're on different versions so maybe that a conflict exists or a bug exists in your version but not in their version so this would be maybe the first step to diagnose that now if you add a hack package name to it so hex info and then package name you're gonna get the most recent description for that package you're gonna get the most recent releases so up to eight of them if there's more than eight it'll show an ellipsis and you'll see it like on this one the first one is 2.0 RC1 the last one is 010 and that's in yellow and it's marked as retired I'll talk about retired packages towards the end of this but that's useful information to know and then the config line is what hex recommends you put in your depths so in this case you'll see that it says it recommends the 1.0 in in your depths so you say like why not the latest version the 2.0 hex looks at the version numbers and when it sees that dash at the end regardless of what comes after it it assumes that's a pre-release version so in this the which you'll commonly see is RC which would be a release candidate or you'll see pre after that but it can literally be anything the internals of facts just look for a dash and so in this case hex isn't gonna recommend you install a pre-release or release candidate as a dependency for obvious reasons so it'll show you the latest stable version and that happens in a few place in hex so just keep that in mind and going further one step further if you add a release number so hex info package then version it's gonna give you specifics about that particular release in this case it includes the dependencies that that package has and so this could be really useful for figuring out why maybe your lock file isn't updating so maybe you have a conflicting requirement on a common dependency so you might say like why can't you know hackney is locked at one one here let's say that you have another package that uses hackney to which I don't think exists but let's just say it does requires hackney to and you'll do your mix steps update you'll find out like why isn't updating to hackney to well this is why you have a dependency in there that in your dependency tree that's locked at at the one one point version all right section two learning all right so now you want to learn more about hex packages so you can just go in your CLI just type in hex search and then a search term and it's gonna bring up right there it's gonna bring up packages that match that result so in this case the search is always limited to a hundred results usually you're gonna get just a handful back that match the version number that's shown there is the latest stable version so it doesn't show pre doesn't show release candidates you can do a multi word search by using a plus and joining joining the words with a plus but just keep in mind right now that's an inclusive search or sorry it's yes it's an inclusive search meaning that it will match on either term not something that matches both terms and behind the scenes this is just a logical and or join on a postgres I like and so there are plans in the repo to put in a more robust search system like an elastic search or react something like that that can maybe do a little bit more but for right now this is this is what hex does all right this is the cool stuff so hex docs online and then a package the this section I'm gonna show you is only available in 0.17.4 so right now unless you're on master you're probably not running that yet but since it's coming out really soon I I wanted to focus on the changes that are in 17 for rather than give this talk today and then next week have it be made obsolete so hex docs online and then package is going to load up the documentation on hex docs for that package the exceptions or some caveats in this is that if the package that you that you're asking for is one of these in this list it's going to look at the version that matches your current elixir version so by default it's gonna give you the latest latest docs or the latest release docs but if you're let's say if you're running elixir 1.5 for instance if you type in hex docs online elixir it's going to bring up the docs for 1.5 not the newest version which I think is 1.62 so and the so if you're in in the context of a mix app and you type this in if that package is included as one of your dependencies it's going to open up the version that you have installed so again like another package it's not just going to give you the latest it's going to be smart it's going to look into your your project see that you're using xyz version and it's going to load the docs up for that automatically and if none of those neither of those things is true it's just going to default to the latest version and I mean this is basically what you're going to see is the hex docs website with the you know the documentation that you're used to so if you if you add a version to that so hex docs online package version then it is going to pick up just that version of it so a corollary to that is hex docs fetch this is actually in 17.3 so you could use this right away what this does is let's say you have a commute coming up let's say you're on a train where you don't get internet access or you're taking a plane but you want to read documentation for this new package that you're checking out or possibly a package that is installed in your system that you just want to learn more about use hex docs fetch it's going to go out to it's going to go out to hex docs and pull down all the documentation files for that and install it on your computer so if you don't include a package name on this right now it's going to download all the docs for all the dependencies in your project so in the context of a mix project let's say you have a hundred dependencies in there in your dependency tree you type in hex docs fetch it's going to go ahead and download all 100 of those put those on your local system so you have them you can archive them you can you know read them offline whatever you want to do with them and so if it's called outside of an app without a package name this is just going to raise an error so you can see like this is one that I did on one of my projects and you could just see it's downloading all these packages there's a if it runs into one where there's no documentation exists so like cowboy for instance publishes their documentation in a different place so that's one of those that you're not going to be able to download documentation for but in this case fetch will just skip over those and move on to the next one so if you add the package name it's going to download the documentation to your home folder your hex home folder so if you look right now in your in your home there's a dot hex directory if you have hex installed and inside there's going to be a bunch of files so one of those is the docs directory so anytime you do a fetch all that stuff is going to go into there so here we go hex docs offline package this is kind of the machination of online and fetch at the same time so what this is going to do is look in your hex home folder for any downloaded documentation if it's there it's going to open up in a browser and serve it up from your file system no internet required if it's not there it's going to download it from hex docs and then open it up in your browser being served from your local file system so obviously you have to download it once but in this case it does the fetch and they open at the same time and then then you have it in perpetuity on your on your computer if you add the latest flag to any of those it's going to skip all those other rules and just give you the latest version you can add that to the fetch offline or the online tasks and like I said it overrides any of that other logic that I talked about all right section three get ready I'm gonna let this run because this is like one of the best all day all right all right this is sort of a more than mundane stuff about hex so I'm gonna go through it kind of quickly but it's important to do because some of the features of hex can't be done until you do this stuff so hex user register so you just type that in to your CLI or to your to your terminal and you're gonna get a few prompts so it's gonna ask you to set a username it's gonna ask you for a unique email address and then to sound set your account password you're gonna need that to do like I said do a few of the things that are available on hacks it's gonna require you to establish a user and once you do this it's gonna send you an email to confirm that address so before it'll work check your email box the only rules to this the username must be at least three characters in length and it can contain letters numbers and a limited number of symbols so underscores hyphens periods and parenz only so reset password local so in hex there's two passwords and kind of two accounts there's one on your local system that's encrypted and one that's on in the hex online system so hex PM those are two separate accounts with two separate passwords same username so reset password local it's gonna give you a prompt to reset your local password that's actually stored in this file here so in your hex home hex dot config file as encrypted key that's the name of it it's encrypted using a 256 cbc algorithm but there's plans as soon as OTP 17 support is dropped it's gonna jump to a 256 gcm which admittedly I know very little about but I think it's better I mean why would they switch it's not better so so yeah reset password account that's gonna send you an email so this is your your hex PM user this is gonna send you an email to reset it's gonna give you a link to the hex PM website to do all there the setting of new passwords when you do this it may invalidate all of your locally stored API keys and so you can see those if you type in hex user key dash dash list it's gonna show you all the the API keys that you have set up on your local user so in this case I have my local user super simple I've got my personal organization and I've got weed maps organization set up so if you ever want to revoke one of those get rid of those keys hex user key revoke and then the key name it's gonna just get rid of one of those keys so if you you know kill your organization if you leave the company whatever you know you can just take that housekeeping clean that up if you do dash dash revoke all it's gonna remove all those so like I was saying when you reset your password on hex PM it is going to or may invalidate all those keys keys that you have stored so generally when you reset your password you're gonna do this you're gonna revoke all clear them all out so that you can refresh them and you do that with hex user auth hex user auth is going to regenerate all the locally stored API keys so rebuild hex config so since I mentioned it your your password is stored in here also some particular configurations that you have if you want to modify how hex works on your system so if you look in that file hex dot config you're gonna see this well something like this and it doesn't make a whole lot of sense but just keep in mind that's where your hex config exists so if you're having problems related to config it may be this file got corrupted or it got lost or something like that so you can technically add any config key you want to that file and it will save it but hex will just ignore it basically so mix hex config key will show you what the value of that that config is and the so you like I said you can use any key you want but the ones that hex looks at are these eight and the one that I think most of these are used if you're gonna set up your own private hex repository so outside of the hex and hex p.m. organizations most of these are our settings that you use for that the one I'd say that is possibly useful to the rest of us is the HTTP concurrency hex is designed right now to concurrently go out and fetch eight eight or send eight requests at a time so when you do a fetch it's gonna send out eight calls basically at once if you find that you're getting like race condition type issues or maybe you have a billion dependencies and it's going too slow you can increase or decrease that number I think you know do that at your own risk basically at the default is eight and that seems to work well for most people but I'd say again just keep in mind that it's there so and again like if you add a value to the end so hex config key value it's gonna set or reset that particular config key or dash dash delete it's gonna remove an existing key alright this section I call defense and so this is this is kind of defending your app so hex that outdated and what this is is basically an audit of your dependencies and this is gonna look at it's gonna list all your primary dependencies what version they're currently on what version is the latest and then if there's an upgrade path available so in this case I'm on 010 which is deprecated the latest version is 103 update is not possible on that it's a major version change however on earmark I'm only a patch version behind so get the big green yes so that means you know it's totally safe to upgrade that dependency and in this case x docs is up to date so green light on that if you add the dash dash all flag to this it's gonna search the entire dependency tree so just because you see three dependencies here each of these three dependencies might have three dependencies of their own and so on and so forth so you may end up with you know 15 or 18 dependencies in your dependency tree if you want to check all of them dash dash all will work and if you the latest version there like I said before is gonna be the latest stable version if you're willing to use pre-releases you can add a dash dash pre flag and that will show you the latest version including pre-releases release candidates and so add a package name to that hex dash outdated or hex that outdated package it'll show all the dependencies that that particular package has so that branch of the tree and and if there's upgrades available and the next part of being defensive about your application is to do an audit so you probably or if you're a ruby programmer you've probably done like a bundle audit before something like that and this is similar to that so hex dot audit is gonna show you any retired packages that you have installed so in this case it found one it tells you the reason for that as well so this is a you know housekeeping thing to keep your dependencies up to date so they're more reliable I'll get into a little bit more about retirement in just a second all right contributing Lil Lil Lil something Lil Wayne right Lil Wayne yeah okay whoo get them confused all right they're all little I don't know all right hex published is anybody here just shout it out if you've ever published a hex package before yeah all right cool so that's great so a couple of things it requires you to be a registered hex user so we talked about registering a user so hex dot publish the command generates and publishes your documents by default if you add a dash dash organization flag it will publish those to the private organization that's not necessary if you put organization in your mix file so if you put organization in your mix file it automatically does this if you don't you can add the flag this way and don't one other caveat to mention about this is watch out for pre-release dependencies so when you go to publish a package it's gonna get rejected if you have any dependencies in your package that are pre-release and the exceptions to that would be if you're publishing it to a private organization they'll let you get away with it or if the version that you're publishing is also a pre-release or a release candidate then your dependencies can be release candidates as well typo squatting another great security feature of hex so we talked about name squatting typo squatting also good so whenever you publish a package on the backend hex looks at the name of that package compares it to every other known package name and does a Levenstein distance calculation on it so the distance right now according to Eric is two so if the if your Levenstein distance is two or less it's gonna send out emails to the hex admins as sort of raising a flag for them they're gonna look at it and determine if it's a bona fide name so maybe it's just happens to be a short name or maybe it's it's it's bona fide and these two things just happen to be very similarly named but it's also a just a really good opportunity for them to take advantage of or protect you against typo squatters so if they say like no this is totally illegitimate it's really just taking advantage of people who misspelled XYZ they're gonna remove it from the pet from the repository the exception to this if you're publishing to an organization hex doesn't check for Levenstein distance so it's organization is your little world you do whatever you want with it okay so let's say you just published something and you realize oh man I totally forgot this or you know I forgot oh it didn't compile well or whatever reason you can do hex stop publish revert dash dash revert and then the version and it's gonna unrelease it for you now this is only available within the first hour of publishing and you know there what this is a really nice security feature so I'm gonna just say left pad and watch everybody cringe and basically this is the response to left pad so once a publish is packaged in our once a package is published in hex it's basically immutable that release exists in that state in perpetuity as long as it's been there for at least an hour and so there can't be a left pad issue where all of a sudden somebody pulled this dependency down and you're trying to build your system and what what the heck happened so hex protects us against that make sure that something like that can never happen now if you if you notice an error in your documentation don't worry about it that is that follows a different set of rules so the immutability is only applies to the compiled package itself so if you type in mix dot publish docs it's only gonna publish the docs in that that case and so if you if you had to make a change in your documentation if you noticed an error you can do this anytime after the package has been released and so there's no limit one-hour limitation on that and yeah so I mean that should encourage you to not have to push out a new release just to push out better documentation so hopefully encourage you to to make your documentation better all right hex dot retire so you have a package and you no longer support it let's say you're just not interested in that project anymore you're not supporting it let's say there's a particular release that relied on you know one dependency and you've taken that out or you don't want to support elixir 1 3 anymore you can retire that package and that doesn't change that immutability law it's still gonna be available in perpetuity but what's it get what it's going to do is raise a flag anytime somebody uses that that you've just required a dependency that's been retired there's five valid reasons that I shouldn't say that there's five reasons that you'd be retiring a package so one would be renamed so when you when you retire a package formally what you'll do is mix hex dot retire package version reason and then message so in this case the reason would be you've renamed it and in which case you should include the new name of the package in the message you've deprecated it if there's a replacement include that in the message security so you found a security vulnerability it's invalid meaning usually means it doesn't compile or other so other is everything else and obviously you want to use the message to to clarify like what the reason is for that so here's an example of retiring and you can also on retire packages so like Michael Jordan on retire and then retire again bad idea all right organizations so and organizations were added in 0 17 1 which was just released not even six months ago it's still in private beta so you can sign up for it on the hex PM website it's generally available as just not really advertised I guess and it was supposed to be released this quarter so it should be released pretty soon in in in earnest and you should expect there be a pricing per user on that so the last price I saw being floated around is about $8 a user and I think I've described this pretty well already but organizations are like your own private firewalled version of hex PM so stuff that just you and your team can see this is the example interface so if you go to hex PM right now if you have an organization like this is what you're gonna see you can add people from your team to your organization you can add packages to your organization and when you join an organization you just have to type this command in hex organization auth and then the name it's gonna authorize your local user for that organization so when you go to pull dependencies it's gonna see that you validated for that it's gonna allow you to pull dependencies from that organization hex organization list is gonna just show all the organizations that you are currently authorized for all right finally join in so I mean hex really is it's an open-source project so it really wouldn't be anything without the people that are working on it and so you know get involved with it so you star or follow the repo on github submit an issue if you think of a feature if you find a bug submit an issue on there so we can work on it or open a pull request to fix a current issue so I know you know I've done a few of these Ivan's Ivan here Ivan has done a lot of work on the hex PM side of things so those are two separate repos and the team behind that Voightek and Eric are you know like Jose they're really really nice people so they appreciate the fact I mean you can imagine there's these people that are willing to build my software for free from all over the world so they're they're really really polite about it and you know if there's something that you're not doing well or if you're just learning elixir their comments are usually really friendly and you know they're willing to help you go in the right direction and that's it so I want to think weed maps they're the people that pay me to be here and elixir days again and if you're interested that's the github for hex thanks everybody