 I know it's the second to last talk and it's the last day of the conch here, so lunch is settling in to keep you engaged Before I get started, I don't know why more folks don't start with the thank-yous First I want to thank my wife and my children for letting me stay up past my bedtime to get things like this done And also the national area closure user group for fielding the first version of this talk There's a third group of folks I want to thank and it's a little more relevant to the talk here and we know the community is full of them And that is paradigm junkies I know the word junkie is a negative connotation in general, but you know closure itself as we know has quite the eclectic background pulling from rich Hickey's Amazon bookshelf and and Michael Fogus's and You know pulling from things to do and some things not to do from other languages And we've managed to come up with a pretty practical platform, right? I think that's part of the reason we're all here We enjoy paradigm shifts. We enjoy learning brand new things that also get things done For this particular talk through a couple of paradigm junkies. We want to thank in particular and that's these guys As we know so for yesterday Dan and will humbled us with quines twines thrines type inferences From any Cameron and David Nolan a community has ported that to closure The only thing you should infer from these photos is that Dan and will you need bigger pictures online that I can pull from I Looked all I could for both of you, but didn't happen So logic programming we've seen a couple of talks yesterday As I said, Dan and will give us a deep dive and earlier today We had a good talk on some of the high-level points in the history behind different parts of logic programming I want to establish a common vocabulary in a common language and really just center on the parts of core logic that I use for web testing So if you go to the core logic wiki, you'll see an example much like this where we're Putting out some facts and relationships and querying on it and I've tweaked a little bit So let's walk through this piece by piece. So they're all on the same page for how core logic works So when we first come to core logic, it's somewhat of a black box and it kind of should be even after you know How it works so it's more important than ever to keep in mind what we're responsible for in our logic program And what the engine is supposed to do for us. So let's step through those our first job is to Create a language that defines how we're going to relate things by default core logic has a few relations built in For lists and other primitive values, but how are we going to talk about the relationships in our application domain? So we start off with doing that and we have these deferral forms that provide for us kind of simple named relationships We're going to talk about men women fun people and people who like people Once we have that vocabulary in place We can establish the actual relationships that we're going to be dealing with and so in this case We're going to hard code some values here that we have some men women Mary and Lucy are both fun nobody else's and Bob likes Mary John likes Martha and Ricky likes Lucy Once we have both a vocabulary for defining relationships and the actual relationships to work with the data We need to do something with that and that's where the really interesting part of logic programming comes in and that is to establish Expectations and I use word expectations is kind of our first key cue over to kind of a testing vocabulary, right? Making assertions adding constraints. So at the bottom here, this run form is our query We've seen this through the other examples that Dan and will gave I'm gonna pull it up to the middle So folks in the back and see it better This is the really important part. I'm just walk through this piece by piece We start with the run form now run is our entry point into logic world outside of which we get to be enclosure inside of which We play by core logics rules We have a star on the end tells the logic engine to give us every possible answer every possible scenario that makes our assertions true Next in line is the queue and this is kind of a convention for having our escape hatch This is our exit point from the logic world whatever happens to get bound to queue Which is a fresh logic variable. It has no particular value at the beginning Whatever we happen to unify with queue or bind to queue in the process of our program is the final return value of the whole run form So that's how I get real values out of out of logic world back into closure world So going next in line here We're gonna set up some nicely named Fresh logic variables in the in this case We don't need to have an extra one, but it's nice to have one that makes sense for the domain We're testing so we're gonna create a new fresh variable person and we're gonna ask things about that person So first there's two assertions I want to make is this person is the person fun and is the person liked by Ricky and Because these phrases are side-by-side and there's no other control structures These are conjunctive such as an and relationship both must be true for the whole run to succeed Finally if those two things are gonna hold we can take the person that was found by that logic run and bind it To queue and again that becomes the return value of the entire run form back to our regular functional programming space And the only person who satisfies all those constraints in this way is Lucy When notice here important is that it's actually a sequence because we have zero one or more than one answers So Lucy is the single answer in this case So we've all done all that work. We had to set up relationships We had to actually you know define the language for them instantiate them and now we you know We had to run a query over them. What is the logic engine going to give us for free? First and most simply and there's a lot more it does than this But this these are the things I want to focus on for our testing purposes I should be able to verify facts that we've encoded in our in our little mini database of people so Here we have is Lucy fun and Q as we said is the return value, but it's bound to nothing in this case So it returns to get the output of a fresh logic variable all the underscore dot numbers We saw that yesterday with the quines just pages and pages of underscore dot number So here we have yes Lucy is fun, but no Martha Martha's no fun so verification of facts and Secondly and possibly more interesting is that it should be able to follow Implications and sometimes people call this making you making inferences that word inferences kind of loaded But it should be able to follow the implications of the relationships that we established in our application So in this case it should be able to work through the fact that the same person is both fun and like by Ricky So let's take a grossly over simplified, you know view of this We know that none of these guys are fun or like by Ricky So we're gonna start with the right answer and walk our way through under the covers core logic keeps You know what's called a substitution map that? Associates the fresh variables along the way with actual values and sees if everything works out in the end We're gonna do it highly simplistically by replacing the word person with the person. We're trying so Lucy is the candidate. Is she fun? Yes, she is Is she like by Ricky? Yes, she is and so finally since she passed through both those phases Q having been bound to nothing else as part of this run is still fresh That's still available to be bound to unified with and so Lucy comes out as the only answer to this puzzle So given all of those behaviors given all that kind of characteristics of logic programming Just a priori what would incline us to think that logic programming is even suited for testing? Why would it make an interesting part of our testing toolkit? I think there are a few areas that come to mind at least for me So the first thing we said about the logic engine was its responsibility is to do some verification of facts in our application domain And we use the word verify What's part of what we're doing with our testing is we're trying to verify that values are a certain a certain value in a Certain context and the question we're gonna ask ourselves is are we already doing some? Modicum of testing by having a relational paradigm as part of our coding. So simple examples to look at cons I know it's closure. Where's the conge? I'm sorry so state cons But cons a onto bcd. I get ABCd nothing surprising there How do we transform this from a regular function to a relational function to a goal? Well, they didn't will told us yesterday. There's two steps first step add an o to the end All right, that's the most important step and as Craig and Dara said if you use a logic program You should you pull in you should alias the logic portion as L. So you can say el conzo el texto el membro It's not entirely culturally sensitive, but it'll be good for good reading So here we have conzo. We don't do things we've added the oh, so it's just a convention So we can keep saying looking between regular functions and relational functions But we've also tacked on one extra parameter here and as was mentioned yesterday What cons would return as output? We're going to tack on as the last argument of this relation and the whole idea is that we're talking about a relationship between two inputs and an Output for the idea of conzing So here we have built in that if I run this so we have q which is the fresh variable conzing a onto bcd Q should be bound to the only possible answer to this and we get back a list of answers with a single answer the list ABCd so If we read a hard code that we can give verification that this behavior is true So, you know, we're already doing so if we're testing cons ourselves like we were writing our own list been testing our own cons We would check that this operation gives that output So there's not a meta question as want us to keep in mind as we work through this is what you know If we're already doing some testing then what's that next level of testing we're going to think about and get into Going beyond that there's some kind of relatively smaller wins that we get as we've seen from all the programs here They're both in this example and from Dan and will Logic program has a declarative syntax and that gives us a win We're trying to encode complex conditions and complex flows that we have a declarative syntax at hand to kind of work through those a little more cleanly Perhaps a subtler point and when we have our own applications and we're doing high-level testing So this is where the focus of this talk is web testing which is high level We're testing a lot of different conditions at once I might step through my application three steps and I want to stop and I want to see you know five different conditions are true 10 different conditions are true and If we're not in by default when we encode as ourselves if else's cons We're adding kind of a sequentialness to our testing. We work through each one in part and we kind of map out the branches ourselves, but As was mentioned before Parts of many Cameron is that better kind of like that Can parts of our program be ordered arbitrarily and for certain things they can't be but for certain simple things they can be So for our original example, it shouldn't matter whether or not, you know We checked that she was fun Lucy was fun first or that Ricky liked her or even if we just bound two fresh variables together Made them you know associated in that substitutions map It shouldn't matter because by the end all the associations will come to make you know the right answer And then finally and this is kind of a more this was the point that kind of spans beyond testing It's testing exploration So at least for myself when working with separate QA teams One of the things that QA guys do a lot better than I do is taking my application and finding different ways to get to The same answer I'm taking different navigational paths so an application or exploring different workflows Which often exposes a brittleness and edge cases that I didn't consider There's no reason why our testing tools get also kind of open the door to a little bit more exploration Especially while we're authoring our tests and so I just want to take you know the example we looked at Here we're looking at a query that's really a combination of a statement and a question and the statement is about Ricky and who he likes and the question is About this fun person who is also liked by Ricky and what would it take for us to transform this program from a kind of It's half statement half question to a full open-ended open-ended question Well, we just replaced the hard-coded value with one extra fresh variable and we're done nothing else structurally changes So here we have person one in person two as opposed to person and we find that lo and behold Lucy isn't the only fun person Like by somebody else. There's also Bob and Mary. So this is a simple example of transforming Hard-coded values statements and making them questions And we're going to find as we kind of explore using logic programs for things that maybe they weren't originally intended for It was we dial the knob between making statements and asking questions a lot of power comes out of that So This goes beyond testing though. I don't need to use this for testing on this showing us we had a custom domain We defined our own vocabulary for relationships And now we have this kind of custom query language for talking about fun people and people who like people and so I don't think this is a great revelation for those who are to do a lot of logic programming But if you've come to core logic and you say well, you know, why would I even bother or you've learned it? Okay, I can do the zero puzzle I can do Sudoku maybe from really good But what might use this for and kind of my more maybe enterprise or you know Genuinely fully practical application and that is that core logic among other things is a query platform So under the covers we already have a query language right in terms of the relations that are part of core logic itself But we can also build our own custom query languages using core logic as a platform for that given the semantics of core logic And under the covers of course many Cameron This is all it takes to transform What would otherwise seem an unrelated domain into a core logic searchable space and there's an example of this on the Core logic wiki of using datomic and bringing in the datomic datums into the ecosystem of a logic program So it's not that hard once you've you know read people's dissertations and listen to David Nolan's talks multiple times Even then it's still kind of magic, right? All right, so we have these characteristics that so April or we haven't done any coding yet We kind of think this is going to work out Maybe there's some things that are going to contribute to the testing story and that kind of core logic as a as a query platform Seems pretty interesting. So move forward, but what do we get? We talked about paradigm junkies at the beginning Where's the paradigm shift? I mean, where's that really big that big win and I'm going to argue that perhaps with using logic programming in our tests We're going to get to a higher and possibly correct in certain circumstances level of abstraction in our tests And we're going to be called to test more and I don't mean that test in terms of number of test cases But actually the nature of tests and what we choose to test So that's logic programming just the crust of mini cannon and for logic, but it's all we're going to use for this talk So web testing what do I mean by web testing for this particular talk? In case it wasn't clear I'm talking about functional testing I was supposed to load testing I'm a system level high level behavioral tests that touch the whole stack as opposed to a little unit tests or integration tests and The final criterion so that I can use the libraries that I want to use is that it should behave like real users when run as much as possible And with these criteria in place there's really one open-source tool that kind of treats this end-to-end and that is selenium web driver So for those who don't know selenium web driver is a suite of tools used for automating programmatically automating graphical browsers So Firefox Chrome IE opera Safari and as I say as a suite of tools There's things like a Firefox add-on for authoring tests There's a distributed version for running your tests across multiple computers with different configurations our focus here is the simple client API is that it exposes especially the Java one and Wrapping the Java is a library called CLJ web driver Which consistently have two API is kind of a low-level core API That's close to the Java metal if you will and then a higher-level taxi web driver taxi Sorry API that is a high-level more terse API for forgetting the same things done And so the basis of this talk is taking those that same set of functions for manipulating a browser that will use for high-level Testing in the first place and combining that with the semantics of core logic bringing it into the logical realm Making our taking some of those regular functions and adding O's everywhere and turning them into logical logic functions or relations or goals So we started with this program and it's somewhat clear how it takes these values And it might put it into a search tree of some kind and traverse that we don't really care That's the whole point What what's the what's the corollary for web testing? Oh, it's the web page. It's all that we have right So how is this going to match up with our we said we had certain responsibilities as authors of logic programs And there were certain things the engine gave us so how's that going to pair up with our other program and the web page? So first what's that language for talking about the relationships between values? What are the values going to be elements in the page browsers browser specifications? How do we relate those? Well, you get that for free right? I wrote those So that's the language. It's a work in progress, of course and contributions are welcome But this sort of gives us a base vocabulary for talking about a web page in a relational manner Next we actually have to have data right no good, but we've already done that work right? We're testing at this point. We have an application. It's full of data Right the web page is chock full of simple things like named attributes and then the nature of the DOM itself is available to us We have hierarchies and relationships that are encoded and available to us programmatically from Selenium and its APIs So there's a host of information there that we can pull from to do our testing All right, and finally establishing expectations. So what does it look like to write a logic program that uses web? Web API is like CLJ web driver. Well, let's start with a couple of ones straight from CLJ web driver I hope these just make natural intuitive sense if I ask for the attribute of an element say the class attribute I'll get the value of that if I ask for its tag I get the tag text text size width and height enabled is it enabled and then Child relationships, you know, does this is a child of the second element? So we said before what do we do first step is? From transforming this from a regular function to a relational function. What's the first step? Add the O's okay. That's mean you're done No, we have to add the O's and then we also tack in the return value of the particular function as the last parameter So now we're talking about you know, what is the relationship for a particular element? It's class attribute and the value of that class attribute and going down the list. So Let's move on from this. Okay. That's our part. We're writing What's it was going to look like when we actually put this to use with the engine? So let's verify a fact about the page We looked at the GitHub login page before the home page of GitHub when you're not logged in it better have a login link, right? Or otherwise not getting in so let's make sure that has a log has a an anchor tag that points to that particular URL So start with run have our qsk patch set up a fresh variable L That's going to be our element. We're talking about Does the element have an a tag? Okay, it does move on and then is the href attribute of that anchor tag GitHub comm so this is kind of a you know a simple way to verify is that thing on the page and lo and behold It went and found it and we have the you know that logic variable output for q That indeed this is on the page So we have an a tag and it has that particular href if I were to change the a tag and ask for a block quote with that We're not going to find a block quote with that and we get the empty list for there are no answers to this particular query All right, so that was verification simple enough. I could have faked that out, right? It's not live coding How about following those implications? So let's take a slightly more complex example We're gonna start here again with q you have two fresh variables So I'm interested in finding a child element of another element on the page So we have L for the child element and parent L for the for the parent So the parent is going to have an idea of header So we're talking about some header div on the top of the page on GitHub We're talking in the neck the child element itself will have an attribute a class attribute of top nav So we're looking for some kind of menu perhaps on the top of the page And then I want to verify is that element in fact the child of its parent and at the end I'm only interested in seeing what that child element actually is so bind that to q and we'll get the answer And I think it deserves a stop here and look what we've done We've now managed to take a whole domain that value see in the bottom commented there is a value from CLJ Web driver representing an element on the page has a ul tag That's the top list of sign up for you for free explore everything else the top of github And we can take this value and pipe it right back into CLJ web driver We could click it we could flash it to see if it's actually there You know it's we've went from closure into the logic world. We did things according to its terms We got back real values So this is extending core logic and it's not it's not rocket science So let's test an actual app or a keynote app. This is the next stage of web development so we have a menu and We have an element with an idea of foo inside of that and I want to test the behavior that I come to this page And I click on foo and lights up and under the covers We're doing that by adding you know in a class of active to that particular element So a very simple behavior Hope we can keep it in our minds as we go through the next code examples We're going to test this twice. We're going to do once just kind of a straightforward CLJ web driver closure tools type of approach And then we'll try to see what happens if we use in logic programming and see what will possible effects that we get So we're going to start by just finding the element on the page CLJ web driver offers us this nice way to just pass in CSS queries and find things So we're looking for the first li under that particular ul. I grabbed that guy I'm going to pull out his attribute and everything was successful. What should the value of the class attribute be? Active okay, so we're going to test for that Using the is macro from closure dot test and we test that is that attribute in fact the string active Okay, we kind of cheated though right because if I were just to test that I clicked the first and only the first head active There's never a situation in a web app when like you click on something and multiple events fire by accident or just like lingering event Handlers or objects and it never happens in web development. So no, we needed to be a little more a little more robust here so let's go ahead and look at all the elements all the list items and To do that we're going to have to take go from element to elements and deal with a collection And so there's multiple ways to skin this closure collection cat But we're going to do it with reduce and we're just going to accept That's the right way at this point So if I were to work through that list of elements checking each one Do you have the right class and if you do I'm going to take your ID and toss it in a list a toss it in a vector and at the end at the end I'll get back, you know the sequence of All the elements that had a class of active and so if this were successful according to our current paradigm We'd get back a sequence of the single item the string foo for the element that had the class active And so we can test that like this Pretty straightforward. Maybe I want to test a multi-select behavior I can click multiple things and they're all lit up And maybe I'm gonna make a simpler collection handling here by using filter for example So I'll filter on elements that have the class attribute of active. I can just count them. I'm okay with that So that was a pretty straightforward, you know only a couple of functions from the CLG webdiver API and some standard closure A collection handling and testing. What's it look like when we're using logic programming? Let's just let's kind of explore Well first a pretty straight translation from the functional approach to the logic programming approach here So so grab that first element. We're going to unify him with q up front Okay, and then we'll just verify that everything we expect is true So does that element have a class attribute that is active now? If this comes back as a successful run if q returns to the element that we're interested in then we know We've won. We know that this has been verified. So this is kind of the first example We've already done some verification. Where's the testing going to come in? Where's the is macro? But we're not quite satisfied like before rock and test just the first element on test all the elements But before we jump into trying to think about closure collection handling here Let's think what we're trying to encode with this particular line We're trying to encode them looking for an element that happens to be an li and it's under this ul So there's a relationship there that here. I've captured as a CSS query. It's inside a string Perhaps a more proper place would be as a constraint in my logic program So this li thing should be just an element that has a particular tag It happens to live under this other element the ul So all of a sudden I now have because I'm using run star and asking for all the answers It's going to work through all those elements So I get the collection handling for free if you will due to the semantics of the logic run So if I take this guy and then we want to test that say foo was the only one clicked I'll just add one more constraint that indeed the ID of the element we're talking about is foo But as we said before we're still we're verifying the behavior, but where is the actual test? Where is the closure dot test is macro? There's a couple ways we could scan this right we could we could instead of having active and foo hard-coded here We could add more fresh variables and get them kind of pulled out of the run and look at those values like we did before You know is it a not is it a list of a single foo ID? Is it a is it three actives? But let's keep it hard-coded so we said before we're going to explore the difference between it You're making statements and asking open-ended questions here. We're gonna make a big statement You know here's all the things that this program should have they should be true about and we're going to test and see if this is true So if this is true, we said we're going to get back one value that that q should be one element So we could test that the run returns a single item in the collection of answers But that's really not what we're trying to test We're not trying to test that they got back a data structure with a single element We're trying to test if this logic run was deterministic if it had a single answer to the query and so it's a slightly different semantic and so we can go ahead and make some helper macros that wrap is and Include the idea of this is a deterministic relationship that succeeds just one time So let's go ahead and look at that So if we add s which is a macro available in web-driven logic, it says this run should succeed once and only once and Again, it's the same as testing if the collection is a single item the and the collection of answers But the idea is we're testing for a deterministic relationship and it's it's one success Maybe I'm testing the multi-select behavior I mentioned before and I can click multiple things and have them all active at the same time I can take out the ID as foo Constraint and say okay, it should succeed twice and only twice still deterministic, but multiple successes, right? Maybe I want to have some non determinism in my app Maybe I'm insane and I want to put randomness you click on an element and things just start lighting up everywhere I don't care if it's you know, how many it is it's just to succeed more than once because I've randomized the output So this is non deterministic behavior Or maybe I want to encode explicit failure The s and the you by the way are inspired from the reason schemer and hash s and hash you as being success and failure respectively Notice in these last three slides I'm going to step back three slides that the body of my logic program doesn't change, right? I have the same semantics here, but I can focus on what is the nature of the success of this logic run? and it's I'm somewhat of a nuanced point, but it's you know, I think it's a Different way to look at it and it's interesting that we can encapsulate all the Relationships that we're interested in in the logic program in the relational paradigm We're already working in and then focus on the success is this enough? No, it's not enough we have to eventually pull out the actual values We're testing sometimes and make sure that for example foo is the only one But once we're thinking about this in terms of the nature of the success of the logic run We can work with it differently when I'm not thinking about doing Collection handling as much as okay, this should succeed and the answer should be exactly foo or Maybe there's multiple answers and I clicked on multiple things and so at least in the answers There should be the items foo and bar that were clicked Or maybe I can just pass in a random predicate that takes in the answer as the first argument and Returns a true or a truthy or a falsi value and that dictates the nature of success so I The combination of things here the focus is We're at a higher level of abstraction We start putting all of our eggs in the logic program or as many as we need to and then And then thinking of for logic as more than just yes at the tool for solver in the zebra puzzle But also as really a platform for writing queries Focused on our own applications so Is it all fun and games? No, it's not this is one of the few times you're gonna actually use keynotes built in graphics. It's great So what are the risks and the first one's a human factor that I think is just worth mentioning pink elephant in the room At least in my experience Most QA folk are not hardcore Linux kernel hackers or working on the closure compiler if you are I'm not talking about you But there's generally a gulf between our QA teams if they're separate and developers and even wider gulf between those enclosure developers And there's an abyss between those and doing logic programming closure development, right? There's a big gap there. We're not talking about this we're talking about teams that are you know Companies that keep developers as their QA and simply switch up teams to test each other's code Or companies that have QA with paired engineers, so this is a human problem a half problem But I think just mention the obvious here cover of the magazine All right performance. This is the really interesting one, right? This is actually perform as the famous as Alan Pearl is quote that lists Programmers know the value of everything but the cost of nothing. I think Richieke has a version of that Let's be honest. There's performance issues up front before we think through the problem So when you're using CLJ web driver or really any conscientious closure API you get laziness by default Right if you ask for 2,000 elements off the page You're not going to get them unless you ask for all of them explicitly You get back a lazy seek of all the elements that match your particular query for example But we're using the logic engine is going to search the page for us if you don't give enough information It's got to go find it right so to search the whole DOM that's an expensive operation to traverse all of that Especially in selenium web driver So we have to think about the generality of our queries that we're passing in they don't ask for every anchor tag If you don't need every anchor tag We don't have any more laziness and of course the page size, but it doesn't take a very big page for this to get expensive in selenium web driver So what's the solution to this particular problem? How can we you know not incur this cost of searching this large space? With possibly expensive API calls. Well the solution that I've added in is poor man's constraints I say it's poor man's because I haven't read the papers that David is now reading on constraints So this is a way to constrain the logic engine to a portion of the page before you ever enter into your run form I provide a facility in web driver logic to dynamically rebind how much of the page gets searched for a particular run outside of the run I mean this gives us you know to sit tell gives blinders to the engine only look at all the dibs or only the header or only the content area Or only the anchor tags And then secondly kind of two layers of control both queries that are top-down and queries that are child or relationships And if you use the library that the tool makes sense as to why they're both there That's these guys So you'll see in early examples in the read me for this for the repo that these guys get used early because oftentimes It's expensive to search a large page for things or to let the logic engine do the searching for you The second side of that coin as I kind of hinted to is that selenium web driver is a factor of the problem I don't say it's a problem because there's so much excellent work put into the project But there are architectural factors that make searching the whole dom via selenium web driver an expensive operation For those who don't know under the covers selenium web driver consists of itself internally of a server and a client in the Server for chrome boots up jetty and it's controls the browser via a driver And your client code that you wrote in Java or the CLG web driver under the covers You think you're making function calls or method calls? No, no You're sitting HTTP requests to this server for everything you do and this is a I mean obviously an expensive operation and working on some kind of Internal closure data structure using regular functions And because of this caching is also a relatively small win for selenium web driver in particular You can hold on to a reference of the element you're talking about the actual Java object But every time you ask for its attribute or do something with it is going to be another call over HTTP to that server So it's a it's a win, but it's a small win and I'm already doing it and seal jweb driver So to be clear queries themselves are fast, you know It's it that the CSS queries you provide or parsed quickly and if you're finding one thing It's no big deal. It's when you start pulling in all the elements and instantiating them as part of that query That becomes a real issue. So what's the way to solve this? Well, there's no way except to get out of selenium web driver So I offer on a secondary namespace in this project called raw and all it does is take the page source from selenium It says thank you and tosses it into in live And now we have a regular set of closure maps with all the all the beauty and the performance of that and the in live selector syntax and there's a drastic I mean unspeakable performance improvement when you use this method So so such a big difference. I have to go back and look as to why CLG web driver has gotten slower And again Hugo Duncan. Thank you for a criterion. I caught that a couple months ago, and it's been great But what do we lose we lose the whole reason we started with selenium web driver Which is it gives us certain guarantees and if we haven't thought through the problem those might not be apparent But if you as a tester are writing this test sequence and you asked to click on an element And it's there in the DOM But there's another div that's come over it in time or it's styled in a way that it's off the page or there's JavaScript preventing you then you shouldn't be able to do that once we get out of selenium web driver We're looking at just the DOM structure. We no longer have those assurances that selenium gives us So if all you need is to verify the DOM, maybe you don't even need to be using selenium web driver in the first place But if you have to for authentication purposes or whatever else you can step out of it for a minute and do things in closure So Is this all we can do as I mentioned before we're using just the thin upper crust of core logic and mini-canon There's almost no it didn't see a condi up there. I can't even do an ore yet, right? Of course you can but it's not included here so the first thing is to know think through how can we fully leverage the mini-canon feature set and And by extension core logic. There's a lot more there that can go in and thinking about testing in the large and using our logic programs in different ways We've explored here just a few of the features there I encourage everyone to go read these things the reason schemer William Burr's dissertation I tried I couldn't finish it and this daunting quotation from Jim Zooey who said at that strange loop that you don't understand Mini-canon until you've implemented it yourself. I Hope that's not true Because I'm not gonna get there anytime soon, but It's it's a point well taken. There's a lot of subtlety there. It's not as simple as we might even think just talking about it So, please go read it. It's fun So in the last thing I want to show and good. I have plenty of time here Is if we take we said before an interesting way to think it's a kind of consider using logic programming for Our tests is to kind of encapsulate all the relational parts of our testing all the information That's related to each other in the logic program and to test in the nature of the success of that logic run But if we do that and the whole thing fails what part of that didn't pan out I had like 10 assertions which assertion was the one that didn't didn't work for me And so this might be a situation where this is definitely a situation where we need better Granule-level reporting and I think I have a couple of ideas of where this is going to go It's not there yet, but I think it's going to involve some deeper diving into into mini-canon itself into core logic But the last thing I want to show so every talk on core logic, you know Thank You Dan and will has to have the part where the program goes backwards and forwards it why So I realized this a couple weeks ago. I was like mine doesn't go backwards and forwards And this to a certain extent right so if you we asked to verify I found a particular element and I checked its class attribute Okay, that was you know I was going forwards and then I just gave the attributes and I found an element that satisfied that that was going backwards But they're not quite backwards. What is the quine of web testing with logic programming? I had to find it so I have a to work in progress Pre-pre-alpha, but when we do arithmetic using closure testing We do something like this five is not equal four and we get this nice output It tells us it failed and what you gave it and what you were supposed to have right and it actually evaluated I mean this there's there's the five what on earth would this look like for doing web testing, okay? So you did something that wasn't the right attribute. Maybe it was misspelled What are you gonna get back from that output? And so I thought through and thought maybe wouldn't it be great if you're running a test on an application And you know your test run failed and I was able to go okay I'm gonna show you what you had DOM wise and what you ought to have had for it to pass So I can give what you had we're testing what you already had How do I generate for you out of logic constraints the DOM needed that you needed to satisfy the constraints you specified in a logic run? So let's look at that This is the one life coding part This is what you were encountering wasn't it? There we go So We're gonna think about this in terms of closure data structures, right? So let's create an element that looks somewhat like what we'd see from it in it bigger, please Okay, I did a test earlier People said it was big enough All right, let's go here No worries Let's see if we get big enough yay nay More okay, okay. I was ready for this. Sorry Yay. All right. There's not very much code I'll keep that for later. Let's make this the size of my screen so Let's define a tag that looks like what we'd see if we're working within live It's a map form I has a tag It has some content and it has adders and we're going to ignore the fact that adders is a not a map right now It's okay implementation specific. This is the this is the client of web testing, right? So I have that guy I can then do the things we've been doing all along So I'm going to ask things about that data structure Good at you do and so I'll keep that guy in there So we're going to verify things about him is his class active and we should get back Can't see the output. That's good. Yes, it is. All right If we were to take out some of these values and ask questions about them So let's keep class and let's ask what is act and what is the value of that particular attribute? We're going to see active. Okay, that still works and if we go the other way and we ask for What is active? Well, the class attribute is active So if I were to then go one step further and say, okay, we're just take this guy out I want you to find me the answer For the purposes of everything else I demonstrated here We'd go to the web page we'd find we let it search the DOM and find an element that had a class attribute That was active, but I'm saying now we want to generate we're going to create DOM for you On the fly. That's what does that look like? That kind of looks like the data structure we specified above I only gave you a couple of the things to verify that the class was active I gave you a default tag and it's a short hop skip and a jump to go from something like that To something like that where you're working on that something you found a div You thought it had the right thing Here's that you needed and the DOM on the right the markup on the right was generated on the fly Using just the constraints in your logic program. So this is kind of where I see different directions He's going. Thank you The quine of web testing. Anyway, that's all I have. Thank you very much Looks like we have five minutes so any questions Yes, yes Simple for me was I was simply wrapping the is macro. I'm sorry. Yes, of course The question was I introduced a couple of handy macros like s and s plus and s as etc. That are macros and the question was why weren't they functions and The reason was that I was looking to simply wrap the is macro And so I built a macro to then you know delay the evaluation to things that get past to is so Is as I said before that's right is s and its friends add kind of an extra level of testing that you had One value that it was between this range that included these items. So it's really just to wrap closure test is the question Yeah, the questions. Yes Sure So the raw namespace that I mentioned actually works within live and in live works with closure maps So with closure maps are the question was Can this be extended to say JSON formatted data or XML and the answer is yes That you know when we already have tools in the closure community for transforming XML into map structures and to kind of traverse over them Whether it's zippers or other kind of declarative things that have been built And so we can definitely take those same semantics and I think we've seen some work with maps and core logic here with another presentation Yes, absolutely in the back in the logic program itself You're saying the question is can we the common the common idiom of having kind of before and after Things happen around test cases and test runs can we encode that in a logic program? And I'll be honest that I envision this part being the part where you've you're kind of done the actions You plan to do and you're stopping and that kind of state snapshot of your application and testing a complex condition But it's an interesting it's an interesting thing to consider And I think part of the work that needs to be done with with the idea is to build it out more into the large and see what it Look like to incorporate the random core logic test run in your in your test suite And maybe there's a place to include Include things like that to make it explicit what's happening before and after a particular logic run So that's a good question definitely open for further work any other questions great. Thank you