 Yep, so I'm here to talk about a project. I've been working on for about a year now called type closure and yeah, I'm Ambrose In this talk, I'm going to be giving some motivation why type closure exists and what kind of problems you can solve with it And I'm going to just jump straight into some hands-on getting started So you can start playing around with it yourself and hopefully with some repel demos that don't go horribly wrong So as closure programmers we like to Verify our code as as being correct and Generally, we need a combination of different techniques to get higher confidence So we've got things like unit testing designed by contract and generative testing and often we use these things in combination So popular one that I use is unit testing and designed by contract. They work very well together. They don't really conflict So one particular tool that's that's not well represented in the closure ecosystem is for static type checking and that might seem obvious because closure is Designed to be a dynamically typed language, but there have been recent advancements in this field of research that Make this last question a good question. Can we build a tool that offers static type checking for closure? So before we wonder about that so why would we want static type checking at all so I Basically found these two use cases that I that I think is it's pretty obvious that type systems can help here. So One is that using Programming and more abstract programming styles like monadic code and Conjure code and these are quite well suited to having static types And another one that's probably more practical is a Java interop because Java's a statically typed language Every time we call a method a field constructor. We have static type information that we can we can look up So that that's really something that other tools haven't taken They don't use that information and a type checker could use this So if I was going to build a type system closure, what would it be like so that the the goals that I had for type closure Was that I should understand most closure idioms and that basically means that we To to port a bit of untyped code to be typed. We don't want to change its structure. We don't want to Rewrite it so it's basically going to be We add we add type annotations and that'll be sufficient To type check it and we also want to take advantage of existing closure code Allow some sort of mixing between typed and untyped code We want to make sure type checking is optional as well and that goes back to my first slides where I was talking about combining different testing techniques, so Basically if type checking is not optional and we fail type checking then we probably want to I don't know boot up a Ripple or use designed by contracts or use some combination of tools So it has to be completely optional to to allow that to happen and Another one of my goals was that I want to run on the the latest version of the closure compiler and not have to fork it So surprise there's an existing project with almost all of these goals for another another language called type racket and type racket is a sister language of racket and racket if you don't already know is an implementation of a scheme and Type racket supports many racket idioms and it provides one of its noble features is that it provides safe interoperability between typed and untyped code and This basically becomes a bit tricky when we when we have untyped code Typed code calling untyped code and we have a higher order function being passed in and then if we get a type error at Run time who do we who do we blame? So there's there's a blame calculus that basically covers all these cases And there's an implementation of this calculus in um type record So that's probably the most interesting thing about type racket And if you want to learn more about that see tam turbine hot stats are awesome phd dissertation So type closure type closure is an optional static type system for closure it's mostly based on type racket and I've put some closure specific extensions and I've mainly built this for two projects my honors project for bachelor of computer science Supervised by Ron Davies and there's a accompanying honors dissertation Which I've submitted for marking and should be available at the end of the month But there is a link in the type closure read me if you want to read it the version I submitted and I also Made type closure a Proposed type closure as a Google Summer of Code project 2012 And I was mentored by David, which I see is in the back now and yeah, thanks David for mentoring me So this is the the box is a narrow reference representation of what's going on in type closure So on the left we have some some code coming in so this is just raw forms lists and vectors that represent Functioned invocations, so we're passing this into the closure compiler, which is the the ugly Java compiler that Chris alluded to so that one particular phase of the the the compiler is the analysis phase and Usually you pass code into the when you when you perform Compilation it goes to the through the analyzer and then you use the analysis results to emit Java bytecode so type closure takes Takes the result of analysis before emission Which is an AST of Java objects and then I pass it through my little Analyzed library, which basically gives me a closure script like nested map representation of the AST and then type closure forms type checking on that that AST and The result is a AST with some extra static type information. I'm not sure how useful that is but yeah Hopefully there's some more boxes that go off to the right in the future So your type closure is just a just a library. It's on closures. There's this github URL so it you just put it in lining in and you just start importing some bars and That's that's basically it doesn't get much more complicated than that So there are two main operations to run the type checker the one that you use the most would be a check NS and that that would Form type checking on a specific namespace and the workflow would be you open up a repel in a namespace and you you would do your type annotations in your regular Over the top of your untyped code, then you'd run check NS and it would check your namespace. So There's also another form called the CF check form and that's pretty handy for repel development You can pass it a form and it forms static type checking or you can give it a form and a static type So what do we need to annotate to get type closure to work? So there are that two usual rules with the things that use local type inference which Which type closure does so the first rule is that we need to Annotate all global bars that are referenced all defined in a type namespace So when we come across a deaf or a deafened in type namespace, we have to already know an expected type And also function parameter types must be annotated and they're never inferred So if you don't annotate the the type for a function parameter, it's just defaulted as any which is the top type It's kind of like object in Java, but it's slightly different to object in type closure And we'll see that later on and many other positions come in third notably local bindings So to annotate a var you just use this an function and you Yeah, I might as well just bring up a repel actually So I'll show you what happens if you try and Check a definition that doesn't already have a An expected type we get this lovely Untied bar reference so it's it's saying I haven't found it a var I haven't found a type associated with this far so we can annotate it and say okay. I want my my function to be a Function that's a number to a number and then when type closure checks this definition it uses that expected type and This is that basically what? Code in type closure looks like it's just normal code in the black there. This is some old core logic stuff that I found might not be in core logic anymore but basically Think to take ways that the bits in blue are the type annotations and you haven't really had to change anything in in the black But of course that you're gonna have to annotate some parts But this is one example of what it might look like in a type namespace So the other important Thing to annotate is these functions So you have two two main ways to annotate functions so you can annotate a complete function type by using and form and This is useful for type checking closures anonymous functions in tax and for giving complete function types that have there both the the the range and the Sorry the parameters and the result type and we can also infer the result in using the fn arrow Fn arrow macro and that basically allows us to just give the types of parameters and the result type is inferred so we can check a An anonymous function so you say one that takes a Single argument and just adds one to it and we can say okay. We expect this to be Again a boring number to number and I should just note that This number is is the Java Lang number and it's it's it's scoped that way because that's our closure is scoped The you have Java Lang star is imported So, yeah, we reuse the the imports That that the current namespace can see and I've actually got a bunch of imports here as well The closure Lang symbol and all that so this is in scope when I when I write my types So let's just run this so yeah, it'll there's also a shorthand for repel development You can also give the expected type as the second argument to CF and also the the function shorthand we can Just say we've got a function with one argument Which is a number and then in further result type There's extra information here, but this is the main bit here It's a function with a number to a number and yet all that the usual stuff is supported with the fn arrow, but I recommend using and form all Almost always except for when you have functions that are so short that you'd probably use the anonymous function syntax But if you are then you need and form anyway, so yeah, it's it's just that very specific cases So type closure supports union and intersection types and particularly we have an undagged union type and a union type is a It's a it's it's a type with a set of functions and an expression is of that type if it is of at least one of the members of that of that union and the the dual of that is the intersection type and if an expression is of an intersection type it is of it must be at least Yeah, it must be a subtype to each one of its members. So I'll just Show an example for that. So if we have a type one We can assign at the union type either nil or number because one is a number then that's a valid thing to do We can also say one is It's the intersection of a long and a number Because it's it's both of those things but Yeah, because one is a Java Lang long But if we say if it's a Java Lang integer, then we get a type error because it's it's not a Java Lang integer even though it is a number so this is some code I've taken from algo monads and It's for those interested. It's the M bind of the maybe monad and it's basically an interesting case of using unions and function types, so It takes it's a function that takes two arguments if you see in the black code there and The the type of the first argument is it's either nil or it's an X and the X here is It's a universally quantified type variable Which means that we read the all XY as for all types X and Y then This so we've got a MV is either a union of Sorry, it's a union of X or nil Then F is a higher order function. So we write that X X actually takes the sorry F takes the X and returns a union of nil and Y Then the return type corresponds to the the body so you we can return either nil or Y so we can abbreviate type types by defining type aliases and We can also parameterize our type aliases by By using type functions, which are basically functions at the type level and For example, we have this any integer alias, which is included in a typed closure and this is basically representing the type that The the types that would return true for the closure core integer question mark Predicate in in closure JVM So Java lang integer long begin begin to just short byte. So it's just an abbreviation and you'd write any integer so a bit of a ad hoc hacky option type, but it basically Models the way that that closure uses the option type is He's using this this type function here So this type function it takes an X and then returns the union of nil and X. So If we call option with any integer then it's just like a function call Except X is now a type so any integer is past as X and Then X is instantiated as any integer. So these two types are equivalent Yep, so the bit more about universal quantification. I think I've covered everything actually But there's a the type for some It it's it was pretty similar to the one we saw before parameterizing over two two types and we can very often infer what these type Type parameters are but if we some reason we can't infer it We use the inst form to manually instantiate a polymorphic type so we can just check check the types that we have already just by By looking up with with CF and is a badly formatted Type signature for some and we can if you just notice that it's it's it's got a it's universally quantified at the moment But we can pass this to inst Just to choose them short types We can instantiate it and all the X's are replaced with ints and all the Y's are replaced with a booleans But usually you don't we don't have to do this and this is inferred using local type inference so type closure uses ordered function ordered intersection types and this is type type racket uses this and basically allows us to to specify multiple arities and multiple combinations of parameter and return types in a single function type and The way it works is when you check a definition with a function type each each arity must must type check for that definition and when you invoke a function that has a Unordered intersection function type it kind of works like a pattern match So if you invoke a function and you know the types of the parameters you're passing you just check each one of the the arities against Against the parameter types you're giving it and the first one takes precedence Over the following ones So This actually becomes crucial for typing things like the sequence abstraction like first So first has three cases With the actual semantics of first oh if we pass Nils a first we get nil if we pass an empty collection to first we get nil And if we pass a non-empty collection we get the the element the first element so the first case of this function section type handles the case where we either pass a a nil or a Or a or an empty collection So if we read it from the left of the red red box We have either a union that is we have a union that is nil or it's an intersection of a sequel X and Exact count zero so exact count zero is the type for the sequences that are empty And if we match this we can safely say it's always going to return nil and nil is the the type for nil in in type closure and the second the second arity says if we have a type that That is both a sequel X and is it a count range one or is has a count greater than one then we always get the the the type that seekable is parameterized by and And usually with a function section type you'd have your most general type at the bottom So it's kind of a catch-all case. So this this is the union of nil and sequel X is more general and than the other ones and it describes all the possible ways to use first and Also the the the result type is the most general out of those the union of nil and X So we have a an inference strategy called occurrence typing which is developed for type racket And one of the key things about recurrence typing is that you can add extra propositions to results of the result types of functions and There are two you can add two propositions in particular You have the then proposition and that is is used if the result of the test sorry if the Result of the expression that you get back is true and the else proposition is used if the result of the expression is false I'll just give an example for that actually first here's here's the seek type and seek uses occurrence typing to To convey the the length of of a sequence so we know that if if we get If we get a true value from seek We know that its parameter is of count range one or it has has one or more count and If we if we get a negative if we get a false value from calling seek we know that either So the way to read these propositions is that the is is a positive proposition. So and then the the the the number of the parameter is on the right so we say either The zeroth argument is nil or it's it's an exact count of zero. So or it's empty So these two functions actually work quite well in harmony in type closure in the way you'd expect so let's make a local binding a and Just to sign it that the the vector with just one in it Let's give it a less general type so type closure can recover what type it originally was So let's just say it's a seekable of number So if we if we test on seek of a we should know that if we go down the then branch It's of count one or more. Otherwise, it's empty, right? So So let's say I just I just want to return some number. So let's say if we're going down the then branch It's going to be one. Otherwise, let's return zero and let's say the expected type is a number so this type checks but we can We can look at what types are actually inferred for for each branch by using print n So print n takes a debug string and then just prints stuff out to the standard output and Let's do it for both Both branches so print of some reason prints everything twice, which I haven't got around to fixing but Yeah, and when I was doing this I found a bug so I'll just explain So we're down the then branch Let's just get a little more When we're down the then the then branch We know that a is going to be non non-empty So the the type for a Down the then branch is actually intersection of seekable number and count range one. So that we've actually got a more accurate type here But the the bug I was talking about is that down the else branch We haven't actually updated the type but we have a proposition environment and we've added these These propositions here here should look somewhat familiar from from the else of seek so Except for saying the zero-auth argument. We've actually instantiated the zero with a so We have the two propositions that a is nil and a is empty so Yeah, right now. I'm not updating the type so so the main thing is that with print n We can do you some static debugging and just check out the types and propositions that are currently in scope so type closure Uses some of the theory from type racket to types to express types about variable arity functions And there are two main kinds of Well, I guess keyword parameters of the ugly one, but the two nice and kinds are Uniform variable arity and non-uniform variable arity So uniform variable arity is where you have arrest arrest parameter that can be expressed with just just some type Like say plus and minus is just any number of any number of numbers But a non-uniform variable arity has a has a non-trivial relationship between its arguments So functions like map and swap bang Have non-trivial Relationships so for with map for example as as the rest parameters grow to the right our first argument to map has to match match the number of rest parameters by the number of parameters you have in your first argument so we can express this with a Theory called variable arity polymorphism But the two types for using just the normal Uniform variable arity we have closure core divide here And the way we express it is a divide must take at least one argument So we say okay We've got a number but we've also got a number star and that's any number of numbers zero or more and and then we return a number and Constantly takes Takes some type X and then returns a type it returns a function that takes any number of Y's and then returns an X. So there's the star again So I'm not going to completely explain how this works But the gist of the type for map is that we have a dotted type variable that's scoped with the the dots So B is in scope as a dotted type variable and that that means that we can put it to the right of of dots and The dots can only Only appear to the left of an arrow immediately to the left so there are two sides to the dots that on the right we have we have the bound in blue and on the left we have the the pre-type on in in red and The subtlety here is that the B is actually in scope as a regular type variable in the red but Don't worry about that The cool thing is that you can do something like this So we're mapping over these two collections And this is just untyped at the moment But we just say we just want to get the first argument and just return it so we'd Let's allocate some types So the cool thing is we've inferred a Pretty accurate type for that and we also haven't needed to redundantly give the type for map There's just one type for map and that expresses all the possible ways of using map. Yes this is a research by Steve Strickland and And Felacin Sam Tobin hot start at Northeastern University really cool stuff So we can express maps with known keyword keys as heterogeneous map types and that the The main thing to know about these types at the moment is that they only record the presence of keyword keys But not their absence. So this buys us a so she associating keys a dissoci Get but not merge because as we merge Say we're merging three maps the maps on the right if we don't know that keys don't exist Then they could overwrite keys that that we want to be certain types so that this this theory gets pretty complicated, but OCaml is just a treasure trove for this and I've managed to avoid any of the The complications my supervisor is at OCaml nut. So he's given me a whole Big list of things to read So I'll be busy next year But we get Here's a type for the the reflection interface for For closure closure has the closure reflect namespace and We have this closure reflect slash type reflect Function and it takes a class and then returns a map with three keys these flags bases and members and Yeah, we can we can define an alias that That represents this map and then we the actual type for the type reflect is Right there, but the thing to note is that I haven't got around to adding keyword parameters So it's just any star, but it's interesting to note that I have heterogeneous map types Which is probably enough to to implement Keyword parameters, so I'll be thinking about how to implement that in the next few months, but hopefully it won't be too hard So let's just show you how this works so let's reflect on the the object class So the type we get back is a is a reflect map And we saw the type for that over here so what we can do is we can we can look it up and we've got a The the type that we expect So what we have we've got get let's get Oh, there was one really cool example actually just give me a second let's find it Yeah, this is cool. Right, so we can so let's get all the members that are methods so we can define an a Sorry a predicate called method question mark and using this this type here it Basically gives you the propositions that if this returns true Then we know that that the first argument is a method if it returns false then we know that it's not a method and then we can give this Give this class methods function function that Takes the reflect map and then returns all the methods So the interesting thing here is that we've destructured the RS, which is the reflect map. We've taken the members Then we've filtered over the members Testing if it's an instance of method Then we've given it a set and then we get this extremely Amazingly accurate type here. So this is all all occurrence typing with a bit of heterogeneous maps Yeah, I think that is pretty awesome That's one of the best use cases for occurrence typing Just the idea that you can recover types in this in this filter is is pretty awesome Yep, so we can give types to data types and protocols and this is I'm not sure of the best way to do this at the moment, but basically with protocols you can annotate each method Even the first argument as you'd like because there are problems between it when you But you want some sort of type inheritance between protocols sometimes because you just you You need to know some extra information So that's kind of a problem. I need to look at as well I won't elaborate on that at the moment, but say if we have a data type a closure reflect field We can annotate expected data types types For data types and other namespaces using the and data type form And this also works in data types in your own namespace But the difference is that if it's in your own namespace then it's going to be type checked But the definition will be type check, but if it's not then the definition won't be so Yeah, so so closure reflect field has four fields has a name type to clear and class and flags and we've given the the types for the for each of the field and type closure Allows us to to form type checking on constructors just the raw dot form or the the arrow form Just thinking I don't think the map arrow form is supported at the moment, but actually the Records don't actually aren't supported at the moment actually Field is a record, but it's also a data type so we can get away with with looking up the field of This I chose the wrong name for field everything is field field field But anyway, we've got it that the the field name FLD name Def there is a symbol because we've we've looked it up and type closure can infer that Knowing that FLD is a field and we've looked up the that the field So this this is one of my favorite bits of type closure the Java interoperability, so We utilize Java static in jadek Java static type information when we interact with Java and that probably the most interesting part is that New is separated as a static type from null so in Java when you have a reference type the compiler thinks great it could be null at any point and You're on your own. Make sure you just don't do it wrong Good luck, but type closure tries to go a little bit further and and separates nil and and reference types and This all works pretty nicely with with the currents typing, which is actually the crucial crucial part and Say we have a Java IO file and we we run We instantiate a new instance of it with the the name a so a constructor just Just normal Java constructors don't ever return null. That's just one of the guarantees that Java gives you and Type closure recognizes this so by default or just all the time Constructors are never null, so you just have to say F is a file and type closure agrees with you sure so by default method invocations aren't nullable and This is because type closure takes the most conservative approach and just assumes That all methods can return null so This is actually semantically correct for the get parent method of F of file because if we don't if we haven't defined a parent then it returns null if we have then it returns a string So this is actually a decent type for this but get name is Always returns its name, so it's always a string so it's never null So what we need to do we need to add it at an assumption to type closures database of assumptions and we use this this function non-nil return and it it takes a Symbol representing a Java method and then So that the second argument to non-nil return is either the keyword all for all arities or it's a set of integers the integers Corresponds to the the arity that you're talking about and then type closure remembers Okay, if we're looking up the Java file get name for any arity, then we know that it's not going to return nil and this is actually a just a static assumption and that if it really does return null then type closure isn't really helping you and This is a good motivation For adding a runtime check for say a debug mode of closure and You could add a null check around this get name Yeah, just to make sure that null never gets never gets called in And we've seen the static debugging with a print end I think I've shown filter set so this Go back to our example here We can also work out which propositions An expression is going to add to the environment if we if we go print filter set Remember how to use it debug string so it takes a debug string and and a form and it to to closure It's as if you just pass that ck, but to type closure. It'll it'll print out a debug string here All right, so here we are the the two propositions that are that are going to be So remember we're looking at this expression here, and we're asking which propositions are attached to this expression and the answer is that we have a filter set with the then proposition of If it's true, then a is a count range one and if it's false then it's either nil or exact count zero And we can also another and form is very useful for static debugging as well So say if we weren't we weren't quite sure that first was going to be a number here. We would add and form Number yeah, might not look like it, but that did statically type check. It's a bit too zoomed in to see but there are actually parts where closure isn't isn't quite It's a bit too flexible to be a type closure to to catch all misuses Say it