 My name is Mark Ingram, I work as a software engineer at examiner.com and I've been involved in the services module since DrupalCon Boston and, as Carl says, it's evolved a lot over those years and we really want to sort of dig in and show you and help you walk away understanding exactly what's there and what's involved in the module. So this is kind of just an overview of what we're going to talk about. You'll be able to answer all these questions when you walk away from this, so what's the services module and what does it seek to solve for Drupal? What are the major changes from 2.0? What is the services server? These are just kind of all the questions that we want to have answered by the time this is all over. So I think we're living in interesting times with regards to web development and certainly the need to actually be able to get data, match data and move data between third party sites is becoming increasingly, increasingly important because that's what's driving the Internet. So the services module basically attempts to provide a extensible API that allows third party sites to integrate with your Drupal site, be it an app, be it another website. Yeah, it's in terms of pulling data between the two areas. So in English we were saying this is the way that Drupal can be made to speak to all the sites. And the key thing as well is that we're really allowing it to build out so that we can do bidirectional communication. So you can perform updates and full-cruel operations as well as retrieving data. So it really is a gateway to access Drupal. One thing we found over the years is that our usage stats are increasing massively. When I first started with the module, we probably had less than 900 installs. We're now up to over 12,000. We're a top 200 Drupal module because there is the need for this. And I think it's testament because one of the key challenges that we're trying to solve in Drupal 8 is getting a first-class REST server into Drupal Core. So we started the process and Dries has made a realization that that's the way that Drupal needs to go to take it forward. Services is basically made up of three key elements. We have the service. The service basically defines what protocols that a site can communicate over and they are basically the primary access point. You'll hear referred to as end points as we talk through the session. Then we have resources. Resources are really the engine of services. Those are really the things that actually do things. So that's why we have a resource that allows you to update a node, delete a node, create a node, retrieve variables. So it really, really is the engine. And then of course, obviously, authentication. Security, very, very, very important. Everything in the Internet. And again, the authentication, the way that we, authentication basically, we have a couple of ways of doing it in core. But the key thing is, as we're going to delve into, everything is extensible. Everything you can, there's altars that you can use to change behaviors, which we're going to go into some detail on. Plus you can build your own, a pluggable authentication methods, resources, and servers. For example, we have a REST server and we have an XMLRPC server. But there's actually a ContraPsoap server. So, yeah, we are really trying to create a framework, rather than being providing everything out of the box. So it's really it's a toolbox. And as Carl was mentioning, and I've seen Clare there as well, I've been around this module for a long time. I first worked with this module when it was Drupal 5, when we actually didn't even have a release. Then we went to services 2, which was like the first true release of services. And yes, things have changed a lot. And they've changed, changed for the better. But I also know there's a lot of people out there who are actually using services too. So I think it's quite important that they actually understand what has changed. Because it's not just a small little change, it's a whole real step change in terms of concepts. Services, when it first came into being, was basically a means for flash applications to pull data back from a Drupal site. And it evolved from there. Then that caused a lot of problems because we didn't really think about things from a real good perspective. Everything was just sort of bolted on. There was a change, a patch that got committed by myself in Paris, which started the process of moving away from resources. That actually went into services 2x. But the problem was there was nothing, the module itself didn't really use it. It was just there and bolted on. And 3 was basically a reworking from the ground up. And yeah, it's really an attempt to actually modernize how Drupal can expose data and bring it into the 21st century, so to speak. Make it actually Web 2.0, rather than something that just did not really work. Another significant change is we removed Kieoth. I was actually one of the primary inventors of Kieoth. And it did sort of work good, but it was our own proprietary system. That was the problem. It wasn't anything to industry standards. And actually it caused probably most of the security issues ever in services. So we ripped it out. And we replaced it with OAuth because OAuth is an industry standard. And we know that those libraries are actually tested and used by many, many more people. So as a result, we actually were providing a better solution for people building websites. And another thing as well is we actually removed certain services. They just didn't make it in. Because again, we had a lot of discussions and we basically decided that we wanted to go down the route of becoming a framework rather than being all things to all men. And that meant that as a result, we could make a leaner, better functioning services module. And we took certain things in. We took a rest server in because, and we made it probably one. We like to think it's a best-of-breed rest server. It can do an awful lot, which we're gonna touch upon later. We also removed the browser. We used to have an inbuilt test browser, which was great. It allowed you to test things. But it wasn't actually using the protocols. So you were never actually really testing what you were running in real life. And again, now, in my view now, we spend a little time on tests. Which again, and now to me, if you build a service and it's your own, build a test and use simple test. Leverage the Drupal framework and actually test it in reality, rather than just having this thing that actually was just executing code in line. And services too is no longer supported. We decided to kill it because it just, it's dead. It's something that just really, really was old. It didn't really serve its purpose. It created a lot more problems than it solved, that's for sure. Yeah, yeah, it's like, yeah. And actually the number of bugs we now get in the issue queue has actually dropped significantly since we actually got rid of two. Because three has tests, things are less broken. So, upgrade. So these are some of the new features that we've got. We looked at authentication and we were like, yeah, I mean, people are going to want to write their own authentication mechanisms, right? So we need to make all this changeable. So authentication is fully pluggable, you can, you can change. You can basically just hook into it and return, I guess, if you return services error, it will be unauthenticated basically. And you can write your own authentication plugins. And we, I think we have a couple examples to show you of that. And Mark did say this, but yeah, we moved REST into core and into core services. And that just, yeah, that just got rid of a major dependency, allowed us to maintain it. And it just, overall, just made it better for us. We added response formats, which is basically the need for a JSON server to let you, you know, so the way that JSON server worked was kind of janky because, I mean, with 2x, you had to do it that way. You had to create your own module to actually create a service. And you don't technically need to do that anymore. So we added in the ability to pass an accept header, like for example, application slash JSON or slash XML or whatever you want. And then it will return it using that response format. And this is fully pluggable too. You can alter the response and change it and interject things. And as I was saying in my previous talk, we needed to strip images out of node content. So we did it on services side instead of doing it on the mobile device because, you know, that would take a lot of processing power. And then we added endpoints, which is basically, it's a configuration object. It houses everything that is your service, your web accessible service. You give it a path and we'll go into, I think we'll go into a lot more about what exactly it is, but we added that into three. And so there's just a couple of key concepts that we wanted to talk about. We built everything off of C tools. Not everything actually. Authentication is not on C tools, but your endpoints and servers are. We, and just recently actually added in library support because we were having problems with spike library and returning YAML and things like that. So this made it a lot easier for us. So we added that in there pretty recently. I don't even, I don't know if that's, is that in 3.1? No, it's 3.X. So that's in the dev that we were hoping to release 3.2 while we're here, but we'll see what happens. Maybe Friday. Maybe Friday. So yeah, so like just some things that we're going to talk about, you know, in the next couple of slides here, just going to answer what a server is and just kind of go over that, go into more depth about resources and how you can use them and actions, targeted actions, basically everything about resources. And then we've kind of touched based on some of this, Mark did, but the importance of security, is it secure, Drupal security team, things like that, and leveraging core mechanisms or OAuth. So a lot of what we wanted to do, and I don't know if we'll probably talk about this when we get into endpoints, but we're really only aiming to support Drupal core. And so yeah, that's just one of the other concepts in 3.X that I don't think is really on that page, but definitely you guys should be aware of that. We get a lot of requests for like, can you write a five star resource? And we're like, why doesn't five star module write it? So yeah, it'll be maintained by somebody who understands the concept behind that module, whereas for us we would have to go through five star and understand how it completely works. So that's kind of why we take that stance on that. But we're more than happy to list modules that use services so that people do know that they exist. Yeah, we're happy to review code as well and make sure that you guys are doing things the right way, because sometimes we'll see people just alter node resources and do things like that when you don't really need to do that, you could write your own resource. So we're more than happy to help you get your things set up on 3.X. Okay, so we're now going to stop just digging a little bit in terms of each of the building blocks that we spoke about. And we've mentioned a few times already that servers, they are the key building block for communicating. And really the first question when you're really setting up an endpoint is to really ask yourself, what protocol do I want to communicate in and what formats do I actually want to accept? Do I want to basically restrict it so that I am only taking XML requests across JSON or do I want to open it up and actually have the ability to do PHP-based posts as well or full JSON objects being sent up? So that's really the first thing you have to think about. Services comes with two servers in core. One is XMLRPC, which now is not really the recommended solution. It does work, but it's had less attention and it really is just a wrapper layer around Drupal's core XMLRPC server. And simply just allowing us to do some extra menu-rooting tricks. So, you know, it's not the recommended one to use. Now, there's many times... Obviously, there's more protocols out there than just REST and just XMLRPC. And, you know, we actually have the ability to easily define a new server. It's a simple case of we have two main hooks that basically are going to be of interest to you. One is hook server info, which is in many ways... It's like really just registers the server and it describes which files is it in. Is there any additional settings that I need to capture? For example, you know, for a REST server, it's what formats, response formats, do you want to actually make it accessible? So, you can inject in additional fields onto your form and then also manipulate and save that data. And that... And also, you've also got the ability there to define a default path. Now, every single path can in fact be alias for a server. So, you're no longer... One of the services 2x was pretty much... If it was a REST server, it lived at forward slash servers REST. Now you can make it a nice clean URL to expose your API into the real world and actually make things memorable. The other thing as well is that you can actually have multiple endpoints or multiple servers all on one site. So you can actually have four or five different REST servers, which, you know, in some ways may sound a little bit strange, but you can then start getting things like... I can actually have a service which is used by, say, developers and, you know, or certain people with privileged rights, and I can keep them separate from my standard day-to-day API which is used by 90% of my traffic. Or I can... If I'm in a web form, I can actually, again, maybe low balance based on which server someone's targeting to make sure we can keep loads balanced. Then there's then hook server. Hook server basically is the engine. That's really where you... The first point where services communicates with a request and decides basically how, in fact, it's going to root it. So XMLRPC is going to go and do any... in terms of handing off. And that is actually all there is to actually creating a new server. Obviously there's more in the sense of what protocol. There's a whole raft of hook server becomes... Yeah, it's for the REST server. It's a lot of code. But that's your entrance point. In terms of request and response formats, yeah, obviously XMLRPC, it is simple. You have one format. You know exactly what you're going to get, which is good in some ways, but it removes all flexibility. It's basically monolithic. Different devices have different preferences in terms of communication. That, again, is why we're saying REST is the ideal solution. REST has many more options, which is nice. You really do have fine granular control in terms of controlling REST. And I think that probably... Maybe we'll be having a barf later in the week, and maybe if someone actually wants to actually sit down and maybe have more of a walk-through, that we can actually take people through some of the actual screen-driven elements as well as just the conceptual sites. So this really is the first building block. This is where the whole services interaction and responses start from. So I'm going to talk a little bit about authentication, which is a pretty important part of your API. Unless you don't have people that sign in, you can just use Session-based and make everything anonymous. But it's a pretty hefty topic, so just bear with me for a bit. It's basically what protects your site. It allows you to protect certain parts of what's going on, and things can get denied based on user access, and that's really what it boils down to in the end. But you can change what function it calls and things like that. One of the things that we think is kind of important is that you need to ensure that you have a good reason to use that authentication method. If you don't know why you need to be using it, there's no point in setting up three-legged OAuth. And in services, we have this concept of if you haven't enabled any authentication, everything is an anonymous request. And there's nothing that you can do about it unless you turn on session authentication. So if you're banging your head against the wall, make sure you check that box. And it's really important to us because it allows us to basically say, you know, if you need to make sure that nobody can do anything as an anonymous user. And honestly, you're at the mercy of your permissions. So if you give anonymous user access to create article content, then that's something that you need to be aware of that if you're giving that ability to an anonymous user, which I don't think most people are, but you just need to be aware that authentication becomes pointless at that point if you give them the ability to do it anonymously. So this really protects us and protects you in the long run. And that's kind of why we did it that way. Yeah, yeah, sorry. We had, I guess, a couple of questions about it a while back and it actually helps stop. We got an essay about it, right? Yeah, we got a security advisory about it. And it's already fixed. This was the fix, basically, is there was a cross-site scripting vulnerability that was able to be done. And I don't remember exactly how it worked, but it was pretty intricate. But this completely solved the problem. So session authentication is basic, your most basic authentication that exists. And it's the only one that exists in core. OAuth does not come in core. It's a separate module, but I'll talk more about that in a bit. It's based on using your Drupal users directly. And what I mean by that is you'll, while with OAuth you would still log in and with the same user, what this is actually doing is setting a cookie and it's an actual session attached to that account. So it goes into the sessions table and Drupal does all of its nice business. And our login resource, which is on the user object, basically uses session authentication. So all of your following requests after you have logged in, when you log in, you'll get a response with a set cookie and you can just grab that and send that with every request thereafter and you should be authenticated. Yeah, so here's OAuth. OAuth is a little bit more complex, but it's a lot more secure. All someone really needs to do to be you is have that cookie, which is kind of bad. So OAuth, what OAuth does, and there's multiple methods, but I'll kind of talk about two-legged and three-legged at the same time. Two-legged is basically what Mark tried to solve with KeyAuth, but it's an industry standard. I think most of the websites use three-legged, but two-legged basically says, I'm going to sign every request with this access key that gets generated in the interface and you give all of your applications that access key. Then it can do anything that it wants as that user who created that access key. That might change a little bit, honestly, because I've taken over the OAuth module for Drupal and I want to make that a lot more easier. I want to make that an easier interface. That might change, but for now, that's kind of how it works. Three-legged OAuth is a little bit more complex and this is what sites like Facebook and Twitter and Google and all of them are using. What it is is you don't really log in on your app. You get sent to the website and you say, hey, I'm trying to access this app. I'm sure some of you have seen this and I need access to these things and so they'll say, hey, we don't know who you are, log in and you log into your trusted site. You see the URL and you know that it's the website that that user account is on and then once you authenticate, it will send you back to your app or site or whatever it is and generate an access key and then you can send that access key to act as that user for as long as it doesn't expire. There's some expiry options that you have but I don't think a lot of people set them and that's a lot more secure because if someone wrote an app that wanted to communicate with your API, this lets the user just feel really secure and it actually is really secure because you're not passing your credentials through some developer's app who could be logging that information and then have access to your account forever. Granted, they can just take your keys but at least they don't have your password. If you're like a lot of the people, they usually have the same password across multiple things. Yeah, we talked about this a little bit. Authentication is not based on seed tools. I guess we kind of just decided there was no reason to. Some people want to be able to export it a lot better but there's probably some things that we can do that. We're really trying to, we want to get off of seed tools but we like the strong arm stuff so we're kind of in the process of trying to figure that out and that might actually honestly be a forex thing. We really don't know what's involved with all of that yet. Is it a pseudo hook system? Yeah. Yeah, so it's basically a pseudo hook system. We have a couple of hooks. Hook services authentication info basically says it's basically a register just like hook server info was and you're basically saying this is the function that I want you to call and I'll call that and if you want to fail out of that you can return a services error otherwise you need to set up the user object and get an actual Drupal user loaded up and returned. Actually I don't think it needs to be returned but you need to set the global user object basically. We don't have an example listed here I think it looked pretty bad on the slide. So yeah, just to get a little bit into resources they are what allows you to... They're like... They're kind of hard to explain honestly but they do everything in terms of... I guess the best way to put it is it's like a hook install file where you define a schema and it lists out all of these things that you want as database tables but we will call them methods. So for example the node... Well yeah, the node method the node resource has create, update, delete and index and files relationship I think I added... I wrote a patch for targeted action on that one but they basically... It has all of the stuff for creating and updating and deleting a node so there's other resources that you can write that don't really have CRUD functionality you don't have to have that it's completely optional I think views doesn't have CRUD so you just have a get view function as an example. So in just types of resources here types of resources that you can define inside your hook resources which is not listed on this page but there's a hook resources function that you define hook resources actually and you can return an array structured with some of these values and it's all documented in services.servers.api.php I think so just to go through them real quick I think everyone knows what CRUD is just create... update, delete and index is basically I want to grab a bunch of nodes it's kind of like views but you don't have as much flexibility as views you can pass in node.title and say I want all nodes with this title or I want all nodes of this type but you can't do things like set a limit and get items 7 through 10 right? If you wanted to do something like that it doesn't work with indexing so that's just something that I think you guys need to be aware of and so actions are actions are basically a post method and I guess I should have mentioned with CRUD creating is a post to your like resource method so it's called I think it's called create so you would do a post request to that because you're posting to slash node update is a put these are just standard things that rest uses and with XMLRPC it's a little different I think it's all posting on XMLRPC but with rest they're different indexes or gets retrieves or gets create is post updates put delete is actually delete and you can actually varnish and stuff can handle the caching because we're using a standard rest methodology for all of this so targeted actions are a little bit more unique like an example of an action would be log in or log out or I don't know if we have any others I think there's like node files and things like that but log in is probably the best example of an action because it's not you're actually doing something I guess you do that with creating but you're performing an action you just say I want to just do something a targeted action a little bit more interesting because it's based off of like an entity so for example node slash one slash unpublish would be a targeted action because you're saying I want to unpublish node one but you need to write that code to do the unpublish so that's they're really cool I wish we used more of them for certain things there's a ton of stuff we could write that would like help you guys we're kind of focused more on just making all of this of framework and like Mark said we're probably going to try and remove the resourcing files and actually make the modules that you turn on and stuff so yeah I could actually do a walkthrough on this of a resource but I've kind of talked about them a bit here so I'm just going to show you guys one one here I hope that's readable so this is our node resource definition right this is just our like I said it kind of looks like hook install you just define a bunch of stuff you can see right here this is our this is what our method is going to be called so this would be like our end point slash end point slash file you know all all those sorts of things and then we have this array of operations which are your standard create retrieve update delete and we just define some things that say hey you know this is where the file is this is the function that you need to call when you realize that they're trying to do this certain thing basically here we're listing out arguments that for retrieve for retrieving a node the only argument that you need is a node ID so name is NID that's basically what your client that you are writing would pass as a parameter we it's not optional because obviously if you didn't send one you'd get nothing back sources is a bit unique I've had some trouble with what exactly this does but this tells for rest it doesn't do really anything for XMLRPC I don't think but for rest this says you're going to grab this parameter from the path and the index is going to be zero and that zero is your end point index not your standard you know arg zero so that's just one thing that you guys should probably be aware of type is doesn't do anything we don't typecast stuff for you it became a huge problem and too you need to typecast it if you can't send it as an array then you need to change your format that you're sending stuff so we just listed here for documentation purposes and maybe we'll do something in the future if we can figure it out but for now it doesn't really do anything it just helps us out I'm probably not going to go through all of these but that's how core defines their services services core defines services we have a little if module exists here comment you know throw this resource in this is our function for retrieving a node so this is what actually gets called when you do an endpoint slash node get request loads the node if it got one it does an entity URI so that we can do you know real paths and then there's some stuff that we've commented on there and then it just totally returns it and that gets sent back to you one of the things that I think I should just talk about a little bit while we're talking about resources is um node resource runs this function right here on line 255 Drupal form submit and I think a lot of people have a problem with that because um different fields between 6 and 7 because it got moved into core handle how you set up the structure for um attaching that particular field or filling in that fields value on a node so I don't think I have any code available to show you but as an example field body in D7 you have to define the language uh you have to define an array with an index of the language and then you have to say I want the first if there's multiple feel if there's if it's like an unlimited field value you'll have to loop through them but if it's only one you know call the first index of that array and then you have to get like the raw value so that's what it's like for text fields but for like image field it's completely different so the format will also vary as well depending on what widget you're using so that's the thing you need to be careful of yeah so like if image field is completely different um file field is kind of the same as image but obviously there's a little bit different there date have fun with that one um yeah so that's kind of what a resource is I'll just go back here and I kind of probably covered some of those potential gotchas um we yeah so there's no there's no standard set of return codes because this is actually up for debate in like every API um we tried to follow the HTTP 1.1 specification for response codes and I think throughout most of the application it's pretty good uh you know if you try to post a note and email field is required and you know throw a 406 you know uh not acceptable because you didn't give me the right information so um if you're not authenticated you'll get a 401 authorized things like that so we yeah we haven't set up anything um that we also haven't yeah it just basically needs to be formalized and another thing that we haven't done which I just thought of was um there's no way to define what fields get returned so maybe you just didn't want all the information and you wanted to pretty it up that doesn't exist in core but if you really wanted to do that it's a simple hook alter on the response and you just unset those fields um so that's just something I don't know if it really fits into this part of the talk but it came to my mind so um yeah and on permissions here again I touched on it on authentication but you really need to make sure that if you have if you have file field permissions on or content type cck field permissions on d6 you know you're not going to be able to save an email address because you don't have access to that field um so you'll get a 401 unauthorized and that's a little confusing but that's what it is you know so yeah we've obviously touched upon all of the individual elements you know it's really like attempt now just to sort of put everything together so you understand sort of how everything sort of steps through in terms of services um most of the actual code we're actually doing the actual um processing of the the real processing of request it lives in services runetime.inc and so in terms of uh so in terms of your endpoint basically what we've got is we start with a menu basically it becomes a menu callback so in that sense it's a normal Drupal interaction that your basic services via C tools create a path that path is stored in Drupal's router table so when you're actually sending in a request to the method node uh update you are actually just there is actually a menu callback which exists yeah and it's just for uh that endpoint we're not setting up menu paths for like node slash zero or node less slash 20 um that's just a parameter that gets passed and it's all based off the endpoint and basically when that request comes in um obviously Drupal roots it through to the to services at which point in time we basically um the endpoint then decides what protocol is actually expected is expected to be being used so at that point in time it's going to say is this a rest request or is this an xmlpc request and it will intelligently hand it off now tomorrow I could swap that server and make it xmlpc instead of rest why I would do that I'm not exactly sure but it would still then re-route it to the new location without you really having to do any other changes um then the next thing we do we do actually do some level of parameter validation even though we don't typecast what we will actually what services we'll actually do we'll say does the request actually look valid does it have the correct number does it have actually have the correct number of parameters being passed in so if it expects four parameters have I got four parameters so we can at least check for malformed requests even if we don't take it any further and check for data yeah it's basically to stop from processing that's not required once we are happy that actually your data looks good then we then go to you know almost the next level of complexity which is then we then go and say ah but are you really around to actually access this resource and the reason why we're doing that is because a lot of times at that point in time you're going to be doing probably doing some database queries in terms of retrieval so you're actually doing again elements that yeah that have a have an impact in terms of resources that are being used on your server and that's server resources rather than services resources by that point in time we have you know we once we're comfortable we actually have a valid user and a valid request we then at that point in time we then like Carl's show the example before that's when we then actually go and say okay now go and get me now go and grab the node that I want to send back and this yeah and everything basically all that we everything we have goes to a common yeah service of controller execute every single server uses the exact same thing so your server is already going to do some pre-processing to format it into the array structure that Drupal is expecting to be sent in the reason why we do it that way is because we want it to be extensible so other servers can actually be plugged into it and then we weren't basically the response the resource will then decide was I successful if I'm not successful I'm going to I'm going to throw a Drupal services error which is then going to be come back in the appropriate format so if you send a JSON requesting you're going to get the error back as a header if you send in an XML PC request you're going to get back in XML a doc which defines the error that has occurred so your end system can then actually process and actually make logical decisions based on the code's returned yeah one patch that was committed last week adds the error to the response body now so I think a lot of people were requesting that and we added that in and that's thanks to Christoph and then once we have the response of the resource then say that then the server is sending the data back so if there is a problem you're going to know about it because we are going to you're going to get a response unless it's a really really strange 500 error and it never actually manages to actually get to that point so at least your app or your website will actually know why I've got a problem and then you can actually dig into docs and work out okay how am I going to fix this yep it's been to a few times I do use Ctools to a small extent and the main benefit is that it now means you can export definitions and so you can happily move them via version control systems and get them from your dev environment into production and again you can version control them as well in terms of actually just in terms of general security from a code perspective there is sort of discussions that maybe want to use Ctools for more things because we're using it already for this one or there's the idea that do we actually still want to keep using Ctools as an actual requirement or should it in fact be moved to become a dependency which you can actually optionally enable if you want this feature everything that we've designed in services has been designed to be extensible it's been designed so that you can basically plug all of these areas in yourselves and we have a vast number of alter functions which basically let you pretty much change the entire behavior of services yeah you can break it pretty badly but the documentation is there for a couple of the functions yeah for example you can alter a resource which has already been touched upon so you could still have your login action but instead of running it through Drupal Core if for example if you're using LDAP then you could then still keep the same endpoint and just intercept that one small little section and everything else to the outside world would look the same so we try to stick to standard Drupal paradigms to allow you to change absolutely everything so you can change resources you can change your controller in fact you can change your request on the fly or it actually gets sent to be executed yeah you could add the parameters to the request without actually sending them yes and then once you actually get the response back you can actually alter them as well before you actually send the data back so that's going back to almost stripping out elements that you didn't actually want to send back in your array structure if you've got certain unique needs or even injecting other data in if conditionally something you know if the node was tagged with a certain term that we're going to do X as well so you really really can make it bend it to your will and yep that actually we actually have an API PHP.API and that actually has examples in there for a lot of these things docs could be improved on the services module and but we still submit docs or we are very very very very happy when that happens and it's been such fun I've just listed a few here at the end there's some examples of all the resources that actually exist out there there's a services views there's a menu service and there's a search service some of those the menu one and the search one you could maybe argue maybe should have gone into services core but we can only take and those actual modules actually have maintainers who are actually interested in making them work so it's they've got a better home than being probably something that might get touched in every six months as someone might find a bug so we one of the problems with services 2x was that you couldn't tell when things were working and when they weren't and we definitely just I have to give a shout out to Yuri Jeresimov I think it's how you say his last name he helped write a lot of these and kind of just fill it all out so good news to him we've got a lot of test coverage here as you can see we have 1743 asserts and I have that note there it could be better for it we basically our process has been when we write a patch we write a test if there's a bug that it fixes we now write a condition that will check that that's fixed one of the things that we did pretty early on right after the launch was we wanted to the arguments that were being passed to every function were to be inside of an array with that parameter name so like your node object had to be inside an array called node and that didn't really make sense it worked it made it a lot easier to manage but it wasn't it just didn't make sense from like a standard real API sense so we rewrote all of the code that handled the argument we rewrote how you got the arguments in and we had to write all these tests to make sure that the old way worked for people and that the new way continued to work or the new way worked better so there's a line in there somewhere that just does a services arc value and pulls it out I think it's right there so we had a backwards compatibility here and you can see that we list the issue ID and we try to be really good about that kind of stuff because it makes it easier on you guys every every patch we write we basically write a new test if it's feasible there's sometimes where it's there's no need for a test because it's like a typo right so or a documentation patch but for the most part if you submit a patch it would be fantastic if you could write a test for it because since they fixed Pyfer whatever it's called our patch statuses can actually be green in services core there was for a long time services Drupal.org wouldn't wouldn't include dependencies so our tests on Drupal.org were just completely broken we could only test locally it was really annoying but they fixed that and we really want to like say that this has helped us so much making sure that everything like just continues to work that if you write your own resources you should probably write some tests for them before you start sending calls with a client because then you can absolutely guarantee that they're working the way that you want them to work and then you can just write the code on your client and and then you're good to go yeah so with third party we had some issues and I touched on that and we're always looking to have people help more I mean we get a lot of support requests now and some of our time is focused on managing that as opposed to new features so if you guys want a new feature just go to town and you know in your own get repo and if anything looks bad we'll you know we'll help you we'll help you fix it and I think that is how a lot of the features got into 3.1 so yeah and it provides a lot of confidence because even you see 1743 passes you know everything's working I would say you turn varnish on and there's like 40 fails you know that varnish is what's doing that and that you need to do something I don't think that'll actually happen but just as an example yep I'm just going to quickly obviously I'll just quickly just talk about this very quickly obviously this is really just a a list just to quickly just sort of seed some ideas obviously 4X is in the future where we're pretty now comfortable as 3.2 is going to be out very soon 3X branch is pretty stable pretty robust now and these are some of the ideas we've got we've got no idea what we're going to do yet that's the thing we really don't we've got a few different ways we might go it's the same as the Drupal community as a whole we know where we want to get to but we're not exactly sure what's going to happen in time and really what we really want is for people join us in the issue queue and actually come in and actually sort of if there's something that you really want to see in services let us know and we'll consider it that's the thing so this slide is really just to say we're flexible we can make things happen if you let us know what you need and one of the key things we've got a problem with is we have a lack of documentation everyone who uses the module knows our documentation sucks we know it sucks the key is we're intending to have a doc sprint on Friday so come join us I said this in my other talk too we know the services module pretty well so writing documentation for it is a little difficult because we know how the module works as opposed to people who haven't used it so we would love some of you guys to come help us out and just be like what the hell do you mean that doesn't work that way you know and so that we can write some good clear concise documentation I'm probably some more tests and there's not just me and Kyle there's actually a vast number of people who have helped us in the past or are still helping us today and just so this is really just thanks to those people because they're as important as we are and anyone who's ever contributed a patch again thanks you make the module rock more we have our own ILC channel do our hashdrupal services again pop in there's normally myself or Kyle's normally around Seattle or Yuri will make comment got any questions you may not get an immediate response because it's quite a quiet channel but you will get a response thank you guys I have a question about authentication we have LDAP authentication use a module called LDAP LDAP in a Drupal but we have CAS for our campus single Cyan solution it's a good way to use the services instead of the CAS module in Drupal I'm not sure I understand the question we have the CAS for the single Cyan solution in campus now we want Drupal to connect to the CAS server so my question is the services module is good for this one services can certainly be used as a single Cyan solution I have used it for that in the past and your issue really you are going to have to roll custom code to do it services is not going to be out of the box and you may encounter some issues depending on exactly how you are authenticating in the sense of are you on the same is it on a sub domain when you actually share cookies there is bigger challenges but services can be used for single Cyan and that would also assume that Drupal is actually making a request to the Drupal site and the Drupal site is actually the second site you are signing on to because services is not going to help you communicate with the third party site it's not designed for that it's designed for taking requests in rather than pushing data communicating with Facebook API so you might be able to do it but I can't really answer at the same time I mean if you can do PHP LDAP authentication then you should be able to write a hook authentication and hook into services and say you know I want to login in this way is that what you want to do yes so I would totally use hook authentication info and look up the documentation for PHP LDAP to authenticate against an LDAP server okay thank you we have multiple questions but two quick ones to use the can you speak to the mic how's that much better so two quick questions if I can to use the session based authentication I'm a service myself but I'm a client to services and I want to use the session based authentication the user based authentication so if I understood you right when I use the login resource and then I will get back a response cookie I have to look for that and then I have to send it as a header I have to send it as a header on every request correct correct second question real quick performance is every request that you make to services involve a triple bootstrap and how fast can you go you know how many requests could you really service if you were hitting it real hard the answer of how many requests is actually that's really down to how big is your box at the end of the day what's your hardware how many lines of code have you got what's your memory footprint so yeah but we do do a full bootstrap it's something that we probably want to try and actually solve we've discussed in the past of maybe having a separate services.php file which basically becomes more of a lightweight bootstrap but there's certain restrictions with that because we have now got dependency on C tools so that it's a complex problem and it's one that we really need to solve and I think that's one of the reasons why at times yet you are going to hit a threshold but if you're using REST and you're actually getting like get responses then you can cache it you can cache it say in a varnish layer so there are ways around the problem and so you're only then doing the initial request might go there or any post so you probably will be worth looking into that as well in terms of actually building up your architecture hey guys what up? first I want to say thank you we've got a production site with users every day we've got a web app a mobile app desktop app uses services works great what I want to know is at the beginning you see services blowing up big time it has been you're at 12,000 users now yeah that we know of we're going mobile everyone knows that your numbers are going to just keep going and I would really like to see a common ground I hate to think that everyone in this room is writing the same resource definition at the same time and you mentioned oh well you're happy listing modules that are using them what about a common ground where we can get together share our resources refine them and have some input from you guys that once they're finalized or approved they then get listed on your page or something well I think you guys have a great framework but we're all doing the same work yeah personally I'm not opposed to adding things to services core that are Drupal core if menu service didn't want to maintain it anymore we could probably take it in and if you wanted to write gosh what's something else you said there's a million things that you would write if you had the time I got the same idea right I have to because I do it for work I don't know if there's a place that we can easily collaborate we're not a Drupal group or something or you're an author I definitely watch the services group and I mean just post a patch if it's done oh we'll gladly accept things that are core and make sense that's a key thing if it's something that's in Drupal core then we will take we will look at it and we will take it you may end it with a few code views to get it chip shaped but we'll take it in normally and if you're into it I got one that's a key question I can't give my cred definitions to work I switch them to custom actions and they work immediately what's the next step that I should take to figure out why I'm getting those 401's you're getting 401's I think so it was like a month ago so you're asking me to dig too deep it's probably a question that we need to talk about in depth so why don't you email us or something and we can answer it alright thanks a lot guys thanks a lot for this module it's been so many years so my thing is I'm trying to do all and I was just talking to Kyle about that and it's a little bit of pain in the butt right now and I wonder if other people have the same issue if you create like a little mini workshop or something I know we talked about maybe Friday doing the documentation I'd be happy to do a boss before then so I wonder if other people are interested to maybe come forward with that request because I feel like session-based authentication is pretty much in the past can you repeat that last part I feel like session-based authentication is pretty much in the past together with that I also have a cross-site scripting question because pretty much when I can get system connect I can get the session ID from the system connect without doing anything really so when I normally call something AJAX I use a token system to get a token to make sure somebody can't abuse my like menu endpoint normally with the token so like how does it actually work with just shot through my head like what I was sitting here like how does it actually work with session-based that I have the security that say that somebody can't just lock me out by like injecting like a cross-site script so you could just call the service without the token I mean just with the session ID right yeah I mean I think that's a problem with session authentication in general just because cookies are not really safe per se but yeah I mean do you have anything you want to add I I mean to say any like because normally let's say when I built an AJAX I put like the get token, the PHP on the PHP side I would receive the token would put it into the request and then on the validation side when I digested after the menu call I check if the token is valid so somebody can't abuse my call yeah I suppose that's really where KeyAuth came from originally that I wasn't happy just with session authentication and I wanted something that could prevent repeat requests so basically a relay attack and things on those lines and actually lock it down which is probably I think if you're looking for high security it maybe is move over to OAuth rather than using sessionAuth yeah that's exactly what I want to do and I'm pretty sure that there's a lot of people interested in that for that reason exactly and that's why I'm saying like maybe people want to come forward and like maybe help and join in you guys with the documentation on that part and you guys maybe just show it quickly how it actually really works and then we can like jump in and like really like help with the documentation everything else and make it better yeah that sounds good again thank you so much for this well are you an ILC or yeah I can yeah thanks guys I appreciate it