 Here we go. So this is the WSTO module to Drupal 8, a case study for Drupal 8. So hello, my name is David Pasco Deloria, I'm the CTO of Cold Front Labs, and I'm spot zero on Drupal.org. So a bit more about me and this topic, I help maintain a lot of modules on Drupal.org. This is my Drupal.org project page, which is a little bit ridiculous. The latest one I'm maintaining now is an entity reference that you probably all use, but one of the modules I'm maintaining is WSTata. That's the one we're going to be talking about. So what is the WSTata module? You probably haven't heard of it unless you've had a very, very specific use case where you needed it. So what's the problem that this module tries to solve? Most of the work that we get asked to do usually involves integrating Drupal content or content managing Drupal with external resources and data, usually via web services. So there's a lot of using Drupal as a portal to a bunch of resources or Drupal loading supplemental data about some item that is also managed in Drupal. And there are solutions for dealing with these sorts of problems in Drupal, but almost all of them are basically the implementation is you point something at your data source and then you slurp it all down into Drupal's database and then that's how you interact with it. It now becomes Drupal content and Drupal claims authority for it so you can integrate it and all the lovely Drupal things like views start working on it. But doing things that way gives you a huge problem which is how you keep this data in sync. So when we're looking at this problem we quickly discovered that syncing is bad. Syncing is a really hard problem and there's no real good way to solve it once you get into the situation where you have data that needs to be synced for remote resource. The real problem with syncing is that you are claiming authority for data that is not yours. You are not authoritative for it but because you're importing it into your database you're saying to Drupal that this is mine, I'm the authoritative source of this data. And then you have the problem of how do you keep that in sync when it's somebody else's data that you are using but Drupal thinks that it belongs to you. So the solution that we found, WS data started as an experiment to see if this would even work and turns out it does, is what if web services are actually fast enough that you could use them as the data source and just load them and keep your remote content remote and let whoever is authoritative for that data just remain authoritative. That gives you a new problem. It means you need to somehow give Drupal an interface where it can understand how to make dynamic calls to various web services to load different data. So you need a system that standardizes and abstracts general web services so that they can be called dynamically by Drupal through various interfaces. So that's what WS data tries to do. Drupal is a standard way to communicate with web services. You can have all kinds of different web services and different back ends and it gives one common interface for calling those web services and then lets you build integration plans built on that. The sort of side benefits that it also provides is it gives you a nice interface to manage web service endpoints. When you write integration with web services you have a tendency to see a lot of hard coded endpoints in various code. This is where this resource is. This gets really problematic if you have different endpoints for different environments. So you have your dev endpoints and your dev interface and your QA endpoints and your QA site, your prod endpoints somewhere else and then you try and export those and then accidentally point to prod and dev. So putting it in code is a general solution but it has problems. WS data ends up giving you an interface which sort of solves this. It lets you export the configurations for your web services into features which lets you sort of manage the configuration. And it also gives developers a simplified API for making web service calls where things are generally handled for you like air handling so you don't have to write all that. Because WS data is sort of written as an experiment that then worked and then was sort of developed into a full-featured set of things or a full-featured module that did all kinds of things the architecture for it is a bit interesting as well as the link that's here for the various components but this is essentially the architecture we went with. It also revolves around how Drupal 7 entities look. We had an entity type called KDA-US Config because they're storing web service config and the bundle for that was WS config types, so the configuration types those would store specifics about the actual source of your data so is this going to be a REST interface or how do you actually talk to it what the base endpoint is, caching support on that and language support that that endpoint has and whether it's up or down or not. Then below the back you have the WS configs which are the individual configurations for that web service so talking to specific services hosted by that WS config type and then those would, when you make a call to a WS, so you could essentially load your WS config and then make a call to it and then it would make a call through the connector which was the actual implementation that spoke sort of REST or just made an HTTP call and then that WS connector could be put around any kind of web client library to match how that abstraction occurred so we had a concept for a WS processor which did the decoding of the response so it would take JSON or XML and turn it into structured data that you just get back so that you don't have to care about it so we can take a look at it, we'll do a quick live demo to show you what that ended up looking like yay alright, so let's make this a little bit bigger so that you can actually see it it's just a Drupal 7 site with WS data enabled you have your web service configuration type I have, let me pull this out of the oven I've pre-configured this one it's Wikipedia so the base endpoint for the web service is Wikipedia slash W which is the web service endpoint it uses our REST client connector to get data from it it doesn't do any kind of language handling you have language handling plugins so it could either change the URL that it calls in different languages or have other ones that parse the output differently to get different languages once we have this we can add a web service configuration so we have a specific method that we're calling from Wikipedia which is Wikipedia read so we have a read data method which is this it's kind of a mess because of how Wikipedia web services work everything is at api.php and then you give it arguments what we have here is in the arguments there is an argument called page which takes a title and so we have a replacement token in there for title and then that will actually be the argument that calls it so there's actually no other configuration here we need it so we have a test setting so we can override how we're going to cache the results of this we can also test it so here we get to put in the replacements that we want so we can say Drupal replace title with Drupal and then make a test call and here we got back some XML from Wikipedia then for what I was talking about before this is a generalized interface for Drupal to call we can write a whole bunch of extensions that you just sort of set up your endpoints and then those extensions can talk to them to load web service data and one of the things that we've implemented in WSdata is a field storage controller so instead of storing field data in a database you can define a field in Drupal but tell Drupal no this isn't stored in the database this is stored in a web service so you actually don't have any tables for this field so you can set up a field like that here we have one attached to nodes the field is installed with data and it uses the XML processor to get that data inside it we're sort of blowing through how this works the remote data key is the data that we want to extract from the payload to get the actual value we want and then how do we replace the token we use the title property of the node and we use that to actually call the web service so in the end what this does is when a node is loaded it will take the title of that node and then populate whenever that node is loaded it will populate the value of the link data field with the result of this web service so if we go to content add content and the link data content we could say I already have one called Drupal wordpress it's spelled wrong wordpress I haven't tested wordpress so hopefully it works so we save it so here's the wordpress page redirect miscapitalization well that's a boring content let's fix that let's change that title there we go now the content is wordpress it's all loaded dynamically when you load the node so using that interface you can connect it in all kinds of interesting ways this is data in D7, mostly does let's go back and do it so I guess I should talk about this sooner this presentation is to me almost like the unofficial tribute sequel to a presentation I saw that at the last Drupal North in Montreal that Adam Bernstein gave and he gave a presentation on how to migrate modules to Drupal 8 and his advice was essentially was not just that you have to restart from scratch which you do because the API is so different but you have to not even restart further than that not even restart it conceptually with the architecture because you still have to re-architect things before you even restart the code you have to restart at the very beginning and restart with what is the goal of this module because Drupal 8 has changed so much just changed so much from D7 that the entire goal of your module might not make sense anymore so you really have to go all the way back to base and decide does Drupal 8 need this feature because there's a chance it doesn't and it doesn't make sense anymore so that's sort of what I did if we look at Drupal 8 Drupal 8 actually comes much better equipped for doing this sort of work than Drupal 7 does Drupal 7 didn't really come with a good Http client you had Drupal Http request everyone's used that function it is especially in the early versions of Drupal 7 3, 4 we limited and handicap as a way to make any kind of web service calls it was good enough to go and get data but anything other than that if you weren't the update module loading XML files from Drupal.org it pretty much wasn't going to meet your needs so in D7 you needed other libraries and if you do a search for modules that say REST Client on Drupal.org you get a whole list of D7 modules that are nothing but API modules for making REST calls Drupal 8 comes with Guzzle which is a really great Http client built into it it's set up as a Drupal 8 service so you just use dependency injection and then you get pretty much to the factory standard PHP, Http client for whatever you're building there's also comes with all kinds of other things that I get into that are really nice but the big problem in D8 is the problem still exists that all these pieces still have to be assembled and it takes there's a lot required if you're going to write some generalized code to call web services so there's still a use case for WS data one of the things I found while porting this is the default behavior of Guzzle is it will throw an exception on any Http result code that is an error so nothing is ridiculous is getting an exception that ends your whole Drupal site because Guzzle got a 404 from a file so it means that if you're going to implement REST functionality, you've got code to write because at least you have to do exception handling so this I guess I'm supposed to write do you really need a whole other module for this because it's better equipped that's the same slide oh let's move on then one of the nice things to do in my Drupal 8 is in WS data 87 everything in all of our plugins are objects we can take a look at them but we kind of have to write our own plugin system to make things easy we set up a whole bunch of interfaces connectors and WS parsers which was a part that decoded the content so that it was easy to extend your own and add your own in them add your own so you could connect WS data to different things we really wanted to we provided a core we wrote a little bit of glue code and then you could talk to and parse whatever you needed this actually made it really complicated for people to use we created our own plugin system so we had to teach people how to use this plugin system and it was object oriented and nothing else in Drupal 7 was object oriented and this is undoubtedly problematic there's no class auto-loadings you kind of always had to make sure that your classes were getting loaded some other way but Drupal 8 has a standard plugin system so that is easy to offer the exact same functionality and we don't have to document it because Drupal.org does so porting actually getting started into the work of bringing this thing to TrueBlight so what were my goals to port this I wanted to make using WS data easy to learn I gave you a run-through of the interface that sort of takes you end to end pretty much you could only do that if you knew what you were doing at that speed because it is pretty complicated I wanted to make it easier to extend which the plugin system does nicely but that was one of the big pain points in the issue queue anybody using this would be like well I want to make it talky animal so how do I do that? well you need to just implement one of these it's like okay how do I do that so definitely one of the things easier to extend we want to add more points so we could alter data flows so certain people that really started using this module ended up being limited by it because they wanted to one last chance to alter their queries and things on the way out just because they wanted to add things to it somebody wanted to add in API keys to things like to specific WS config calls like as they were leaving but before another step so that was there wasn't an interface to do that one of the things we wanted to get away from because it was really confusing is the custom terminology that we ended up implementing with WS configs and config types there's not really a good nobody nobody would know what that did by the name I mean in sort of you have to really explain every single step I kind of wanted to reduce the amount of boilerplate code you need to actually use this and I'll show you what I sort of mean by that I don't know and we definitely want to have more examples and tutorials and documentation so that it's easier to get up and run it so when it comes to re-name so when it comes to re-naming the components basically the name we did is WS call it's actually making WS call the config types become server, a server that you're making a call to and then the WS parser became decoded but now we've got two points to encode on the way out and decode data on the way back we re-huller our key factors to resolve some of the build system's pain points to give you less to config it there's still the two sort of items you have to figure but on the WS server you choose what connector you want to use which is the implementation that should talk to your service and then consider WS call and then that one you sort of choose your encoded decoder and what service you are or you want to connect to and it sort of makes things a bit more straightforward the thing that's coming up with all these graphs is they make nice documentation better development API so here's an example of what code looks like you can call WS data and you can set it essentially what you're doing the first line loads the configuration entity and that second line calls it which goes and makes a web service call and brings back some data it's unprocessed so you have to load your processor and gives the processor your data parses your data and then you ask the processor for your parts data and it gives you your data structure that you want there's no sort of error handling in this code but this is sort of the minimum boilerplate you need in D8 we sort of felt that you should be able to configure all of that you didn't need that code you're only going to parse the results of a web service call one way or else you should make a second call that does something different that's the whole thing for you so in D8 WS data is actually a service is that when good with what services are in Drupal Lake? yes? wow okay I can talk about it if you want did you want me to talk about what services are in Drupal Lake? oh no I don't okay good that's awesome so it's a registered service you don't even need the first line you just say that WS data is a dependency and then you will get the service, the dependency injection and then it comes in one call you just say WS data on the service call and then whatever your WS call machine name is and it will give you back the parse data directly so it makes writing web service connections pretty straightforward so the other thing that I have found maintaining the D7 module is that regardless of how much documentation you write or how you sort of talk to people about the architecture things people always ask you for more and more documentation and the reason is because they don't want documentation they want simple examples and they want tutorials people need to get pretty desperate or frustrated or something before they read the documentation for it so asking for documentation is not necessarily pointing them at the documentation isn't actually helpful they really need specific examples and in a bunch of different ways of doing sort of similar things they can understand how things work people are very want to say visual people learn how things work by watching them work not by sort of describing it alright, so that's sort of my basic what I wanted to do reporting this module so what did I did all those things so what did I learn reporting a module to Drupal 8 or this particular module to Drupal 8 so the thing that I learned Adam Bernstein said his presentation was very true you really do need to start from scratch what's given to you as a developer in D8 is completely different I'm very impressed that they managed to make the site building interface in D8 and D7 look so similar because what powers it is completely different it is just a completely different architecture doing a real, setting up real planning and architecture and sitting down thinking about these things and D8 really, really pays off these are some of my early really ugly sketches of how I was going to set up the different components and what those components were going to be WS calls and WS servers were going to be configuration entities I did some experiments to figure out what kind of entity should they be what would be configuration entities and then WS connectors and parsers were going to be plugins and so really working with Drupal 8 it was really beneficial to do this first get an architecture for what these objects are going to be figure that out make sure it all makes sense and try and build them Drupal console Drupal console is a really neat tool most of what it does is co-generation for things it will create new entity types for you new configuration entity types new content entity types new plugin types and it will set up all the little files for you I was really worried about using it I was not going to learn anything it was going to do it for me but Drupal console is really not a silver bullet like that it is really really useful to get your bootstraps with your objects so once you have your architecture done I went to Drupal console and built a whole layer of what I wanted the module to do but it didn't do anything yet at that point other than register all these objects in Drupal and give me some basic QIs which would have been kind of a pain to set up and in the end I ended up having to essentially touch every file Drupal console generated for me and put in code or changed things essentially nudge things at the least nudge things back and forth in a certain way so I had to go through it anyway I could go through all these files anyway so I did have to learn all the things it did so it was actually a really useful tool to get started on you would probably have a lot of trouble with it if you skipped your architecture set and just wanted to code something because it makes you decide the kind of thing you want first but if you do architect it ends up integrating nicely with that so the other thing the plugin system in Drupal 8 I really really like and I really understand why they used everything in Drupal 8 is kind of a plugin when you get down to the core of it it regards how you feel about annotations you can also YAML annotate them to load them in the plugin system is a really great way of adding stuff to Drupal 8 and registering it and with Drupal console setting up custom plugin types it is really really easy and really fits in with the object oriented model that Drupal 8 provides so if you're going to have an object that sort of does something and that something could be generic anyways it's almost worth making it a plugin because then anybody could sort of write a plugin that does whatever it is like differently and then just use that instead it's definitely worth looking into something else is use the coder module frequently the coding standard the coding standard for Drupal 8 is different than Drupal 7 because most of the things architecturally are very different but the coding standard actually changed while I was putting this module there slightly so that was fun and for a while I just kind of left it because it was changing and then went back to doing the coding standard on it and had a whole lot of issues that I kind of had to resolve but instead of waiting to the end I definitely run coder the coding standard checks frequently they end up finding a lot of stuff that you wouldn't that are not coding standard necessarily violations but sort of interesting things like oh you have this variable that you use only once so it probably doesn't do anything finds a bug code and things like that and the coder module also comes with another coding called Drupal practice and Drupal practice goes to your code and tries to evaluate whether you've done things in the best practice way the one that really comes up came up a lot for me is that it didn't like that I was calling Drupal services directly it says you should be using a dependency injection for this so I was like oh man I figured I had to use dependency injection on this particular item I don't want to use it here but it actually made me learn like what are the there are base classes that handle dependency injection for you in Drupal core so you can just inherit from them and then it's really trivial to add dependency injection to custom stuff so it's actually worth following everything in Drupal practice and making it do that yeah like I said there it does a good job of training you so this is a problem that I ran into Drupal 8 tries to cache everything everything that you do by default which is kind of a change from Drupal 7 Drupal 7 sort of page cache things but like Drupal 8 caches every piece you do and tries to do it by default I was running into this strange exception I had when I was writing the form that lets you test because what it would do is call the call method which would say hey somebody's clicked the call button make a call to this web service and put the result on the page so we're trying to form rebuild or tell Drupal to rebuild the form so it could put the result on the page and the form cache would throw an exception and the whole thing would die and it's like but this happens outside of my code what is going on and the problem is I'm jamming uncacheable things into a form that the form is then trying to and the form's not the cached itself and then tries to cache itself again and it already existed and so it's causing this exception but really my problem was I just needed to tell the form that this form does uncacheable things and don't try and cache it and the form has an effort that says disable cache I just needed to call that and it didn't really work but it's just something to keep in mind something I ran into when courting things to de-age the other thing that I started sort of touched on before in my own stuff but really started on this and now I'm due for all of my architecture there's a language called plant-uml I'm not sure how many people here know remember their uml but there's a language called plant-uml that is just a text-based language sort of symbolic kind of like graphed is way, way easier to use and it renders uml different kinds of uml diagrams it makes building diagrams for things trivial and then you can commit them to source it is really really worth learning and implementing through projects that's how I did the component diagram in the previous slide and how I did this sequence diagram which is part of the documentation including the module so somebody really wants to dive into the documentation they'll be like wow look at all these graphs but it's actually sort of really really good way of getting their thoughts down about how these things should work and how these things are connected and you can put it down into code pictures for you that you can show off the sequence size diagram is essentially what happens when you call call the call and put it on the service so it goes to the service I don't know, I'm not I'm going to go through the whole thing but uh, yes plant-uml is a really great tool maybe I'll show you what some plant-uml looks like if you're interested so it uh essentially for basic yes I got you, you're wearing cookies let's zoom in okay there we are, here is some plant-uml you start with start-uml and then point out it goes to Bob Bob goes to something else and it is really really straightforward it's like really fast sequence diagrams or any other class diagrams or state diagrams or component diagrams it makes having really fancy documentation easy alright, yeah so the other thing I found too is that services are pretty useful while writing different things I've used a lot of different services that are exposed by Drupal Core and write my own and expose my own service they they're actually they're actually a really useful tool in terms of exposing functionality to users and it really does let you sort of force people down a certain path that you want for that functionality because you can sort of block access to other stuff and be like no, no, no if you're going to use this functionality and then you can make sure that all of your validation follows a specific path in D7 you're generally creating a whole bunch of these global functions and then if you didn't want people to call them you'd start them with an underscore but if you're writing an API people are going to be looking at those functions and they might call them directly if they want to do something specific but that could get them in trouble in other ways that they might not be aware of but this lets you really provide more of a complete interface to expose alright so what was the result of all this so the status, there's an alpha 2 which I tagged and tried to create last night but the gifl.org was not seeing my tag for whatever reason but then that was I was going to fight that for that long where we are now is the core components sort of all work but the integration components need to be written I want to write integration components for triple console so that you can automatically have triple console generate the shells of all the plugins for you the WS data 57 has a whole bunch of trash commands so you can script deployments so if you deploy to this environment it will sort of disable these web services so that if something calls them it will return false immediately which makes sense for certain environments that don't have access to certain stuff or you don't want your dev environment to be able to access your email service or something like that it can also change the endpoints with drush so let's script a whole bunch of things to deployments that isn't there for the V8 the field controller, field storage controller that doesn't exist yet that has to be written the thing I also want to add is more rules integration there isn't rules integration in D7 which I should probably write but it would be pretty cool to have in rules to be like when this happens make this web service call and introduce something else and then you can essentially have Drupal push data to other services and then the general block implementation so you can have a block that essentially you say this block renders the result of this web service and you can have a block that just you stick on the side of something that loads something about it so it gets in context and loads the say there a Wikipedia article that's similar so that's what I need to still write so all the good stuff that doesn't exist alright so we can take a quick look at that and hopefully it works alright so same thing it's under structure except now it's a web service server here's the Wikipedia connector very similar except now things are a lot more simplified just have its name tells you the machine name the base endpoint is and then because it's easier to write connectors and because Drupal comes with an HTTP client you can take on packages with a lot more connectors so you guys probably can't read that there we are the pre in the Drupal server version it only came with a REST connector because it was really designed around making RESTful the full REST support like correct calls to fields for the build storage but that isn't what most people including us ended up using it for the bit most of the use case was making simple HTTP calls that you wanted to configure it turns out that even in the RESTful world every single web service is implemented completely differently so you need to be as flexible as possible because there's web service standardization even in a web service standard so the more you can configure the better so I still left the RESTful Connection there which is what I'm actually doing now you can also set it to a local file directly so it'll load file I get files from the file system and you can interact with that instead or just a simple HTTP connector and then simplify it again RWS calls so you can it has its own machine name and what it's label is you can pick what server you want to use for it it'll look at the server and then generate the RESTful form to give you the options for what kind of connector that that one is connected with if you choose local file it'll just say what's the file name that you want to interact with but this one says what's the path and what HTTP methods you want to use to call it how you want to decode the results so let's take the XML decoder and then how I want to encode data I send to this I just don't want to do anything so it'll pass strings as is and then you can test it and because of the way so this just tests it and runs it like this so this is the XML it's returned I'll have the code for this hold on a second sorry guys actually let's just go to the slide there we are bad developer api live code of course it doesn't work anyways instead of trying to debug why my mic php eval doesn't work we're just going to move on but yeah so this is a simplified version of d8 and what the d8 version looks like oops I'm a fire indeed alright and that's it so are there any questions hey I got a big one sure so I understand that d8 they re-occur as services so they're very internally focused would you say that in the future that they would be as you've done focused like the core team if you focused on integrating the design code services or would we all have to adopt services and they would see that all developers are the community aspect of all the developers that are doing this like how does this functionality get into the core because you talk about cash issues and stuff like that the core would kind of understand this stuff yeah so what Drupal is for at its core and everything in Drupal is really about I'm the content management system and I'm the only content management system here so any data interact with his mind I know the data model so I can offer things like views but it's my data I know when I change my data my data won't be changed another way and that's why you can get into such big trouble and get such weird side effects if you go and edit the database content in the database directly you can make weird stuff happen because Drupal sort of depends on you going through each data flows to alter the data because it does things like clear certain caches when it knows data has been updated and if you circumvent that it gets angry at you or you just run into bugs so really like that's what Drupal sort of how Drupal thinks and what its goal is WS data lets you do something to attend gentle to that so I would never see this functionality as something that would go to Drupal core this is definitely like a contrib thing what if you want to make Drupal play with sort of things that you're not managing content you're not managing and integrate that with your content management system which really sort of leaves the scope of a content management system you can see where my decision tree is going architectural like is Drupal the front end you know for a large corporation you have many different services that are even built across the corporation what's the front end to that is that Drupal a Drupal website consuming those services or is Drupal just one of those web services and then something else being the front end is consuming all those services but if you're going to look at it like this then this is essentially the reverse okay so and you wouldn't do that that's crazy because Drupal is your content management system and the rendering system is not necessarily the part of Drupal you really like and want to keep but so it's really just used for extra data so you could do both if you wanted to have sort of a headless Drupal instance and write something and react that loaded data from like loaded a specific sort of object of data that to render but in your back end the object could be composed of multiple pieces of data from different places but most of it is in Drupal you could use this to sort of make those objects that Drupal is presenting to your react front end be like one consistent object as opposed to having your react engine have to talk to things and then assemble it itself so that would be the use case in that if I think I got your question alright cool what architecturally of course you're thinking of use cases but one of our approaches for remote data the past has been leveraging some solar cores to index something and I guess there's pros and cons evidently to that looks like you can tap into this being a node in the end so you could do other things around that web service call where I guess I think solar data that is not always a node yeah I guess there's a big evaluation criteria there has this potentially worked with Siemens or is there any scenarios around documents and just concerned about performance sometimes I guess it's the use case it's really use case this is really so you can use it with anything and customize it I mean kind of end up in caveat empty territory when you need performance so when you're generally doing things with solar you end up with different problems well not different problems but you end up in a different place because solar presents a really standardized thing and if you know that people are indexing data in there you can get like a data model out of them that's pretty standard and so you can do cool stuff with solar integration and a lot of new modules do that like with Sarnia and well search API and things like that but if you look at the way that those modules work search API, Apache Solar, Sarnia they're all about most of the work you have to do is to explain the data model of the data in solar to Drupal and that's all your work you tell Drupal about the data model where certain things are and where they go and then what they provide is an interface to make the solar queries the data model to get the data they want and sort of how Drupal goes and the interface between the data model you provided and Drupal so WSdata lets you skip that step by sort of being like well you know what the data model is so just configure web services to do that and we won't ask you to tell explain your data model to Drupal so that Drupal can can close it to all these internal tools this is just sort of plug in what you want your actual calls to be we don't care about the data model we'll just give away for Drupal to load this data so yeah that's really where this sort of fits in you can turn on you can turn on I guess commenting and rating on nodes now because you're in Drupal I guess it makes me think of some of the stuff they went through with open data yes yes so this would be sort of another way of doing this similar things to what they did in open data the so the benefit there is because they're only talking to one place they can really get benefits from taking the time to meticulously explain to Drupal what the data model is and how to get all these components and how to interact with them and how to query them and how to query different components and WSdata assumes that you're not going to do that and so it's going to be limiting in other ways and freeing in different ways yeah you could absolutely do that over there as a follow-up there's a module specifically for search API that's you the few data that was in that controller it has some other process so if that's your specific use case it's the same module right search API do you need some sort of data or something oh there's more okay so that's your specific use case in WSdata it depends what you're trying to do when you start getting into this kind of territory things get complicated enough where it's worth looking at these modules yeah like if you're going to express to Drupal what your data model is start talking forever these modules but you really have to look at the benefits of like what you're trying to gain if you're trying to expose your data model you have a remote engine to Drupal so that you can make one view like maybe don't do that and just get the data and render it in a real fashion way everybody else doesn't that aren't using Drupal and don't have to use so it's sort of evaluate your requirements like what's going to be cheapest and easiest to maintain for your team and that kind of thing it's complicated enough where you can't be like this is the right solution to these problems because it gets really complicated it's really fast yeah exactly yeah it's kind of the same way to this question we talked about the data that you're using so far if I'm so much saying it's like you're using these three very structured data in a specific way and this is more so it's not even that because you could have self-structured data and prefer this over external entities it's what are you going to do with them now that they're entities so you've got these external entities like what are you trying to leverage in Drupal how did they making them entities make your life easy but they still aren't doing anything so what was so great about making them entities and there are some great things about making them entities don't get me wrong I don't want to disparage entities but entities come with this baggage and functionality so I have to look at that is this what I'm going to do with this functionality and can I live with this baggage and depending on your use case sometimes it can sometimes it's worth it sometimes it isn't and you know when you're doing your architecture plan that's when you make these decisions okay can you tell us a bit about views integration about views integration I sure can, so views is an interesting beast views is really where Drupal misunderstands the data model behind things because that's what essentially a view likes to do is build a SQL query oh who doesn't like that it stopped recording it stopped recording does this whatever so views really likes to change I don't know how much time it might be so flag me if you guys want to escape because views build a SQL query to select a certain number of things so you can get in trouble if you just want to filter by something that's remote because how do you do that so you've got a SQL query that gets you everything and then you then window down afterwards and then then paginate or what if you want to filter by something that is a data property that's in the data and something that's remote so views is a very tricky thing and the implementations that you use well force you to only use one data source so if you're looking at the standard views it will build you a SQL query and that's how you're doing your filtering if you're looking at search API views it will build you a query and that's how you're doing your filtering and you can't sort of mix and match things if you want to filter more and you've got to index it in the query and same thing if you're using like a EFQ views then you're restricted using your energy filter query to do your filtering so there is use integration for this but it's it's sort of rudimentary because it'll just show fields and when you sort through them and filter them we can only filter by the results that are returned from the database the way that we've solved this problem is because of the nature of when this data is loaded on NUB low just use search API filter just use search API so index everything make a view on that index and then all of a sudden everything works because from the node view point when the loaded nodes point view these aren't database fields or remote fields it's just data in the energy so if you're indexing the energy you can index that to search and everything will just sort of match okay anyways if you have any questions please come talk to me and yeah, thank you very much