 Hello, everyone. Thank you very much for coming to our talk. We're about to start Here we are on our second Drupal con in the United States last time We were visiting the first Drupal con of our company in Drupal con Austin, which was a blast And we're very happy here. We're very happy to be here today to show you one of our most beloved projects that we've been with we've done in the last months And so our talk is about MercadoIlegal.com or MercadoIlegal.com Which is a crowdsource reporting web app based on a Drupal headless installation and that's what we're going to talk about today So I hope you enjoy it and we'll have of course Q&A at the end which Hopefully we can share stories from horror stories from exciting stories and everything in between from your Experiences using headless Drupal, Angular, etc. So that's good. Sorry. I Am Alberto Rojas. I'm partner and co-founder of our company Manati You can find me on Twitter and you can always go and check out our website I'm here with my friend Andres, which is a team member from our company as well He's a PM and a front-end dev that worked heavily on this project And he can give us up great insights on what we did So like I told you We're a company our company is called Manati We are from Costa Rica and we are web agency. We specialize in Drupal We like to do fast loading websites cool looking websites And we like to make the web a better place and we have a team that it's Plenty diverse I think We are 12 individuals with different backgrounds ranging from arts design Electrical engineering computer science psychology and philology communication. So we can take on all these projects and give Give value to them not just on the technological side of it, but we can work on Some more stuff that every project needs. So Let's go and take a look at what we did with Mercado Legal This is an app That is very important right now and let's put it in context first It is a private initiative from AMCHAM, which is the American Chamber of Commerce and the Ministry of Finance of our country They made a partnership and created an app so that individuals can be empowered to share Information about trafficking illegal goods drugs, you know, whatever it's being trafficked. You can report it with this app They create a video that explains Better what the project sold about what their what their initiatives all about so let's take a look Only in a year So now that we have a little context on it we can take a look at the project Whenever you start a project a project of this nature you have a lot of assumptions even on the technological side and That is one of the greatest challenges when you're working with something that you don't necessarily know or you know How to approach it, but you have never done that before and that was our case We have already done web services and Front-end for another application for other applications that are in Drupal But nothing mixed So it was kind of risky from us to do this whole thing and we have we had a very tight deadline So let's talk about challenges There are always challenges here and we have a very interesting Architecture on this on on this site actually Drupal is a backend for multiple and month multi-lingual sites It's not only the Drupal edits for multiple sites, but also the front-end and we're having only one single code base that serves The number of the whatever number of sites that can be created Those sites are customizable sites and they live in different domains. So that is a challenge You have to give them branding colors and also different names and there's also a decoupled responsive front-end on a different server and Calling it server. I think it's a more than what it is because since it's so lightweight. It can live on a very simple Server or a hosting account. So we tried to make it to make it As best we could so it could have a very thin layer of infrastructure There were other challenges, of course With very specific functionality that we needed for example, we have of course if you've seen the app before it's Heavily based on maps Since if you know about Angular, you know that shareable content. It's a It's not that easy because metadata is not available for you At least that's where you expected it to be so that is a challenge Is it to use front-end? for people that are using these these application because It's not us that are creating the content because it's our users that are creating the content They know where the information is and they know where the contraband is taking place so They have to be able to do that and also the site needs to be easily Customized on the back end. So those were one of the main challenges That we approached on this project and There's also surprises Something that you don't expect at all In this case we had Very good UX designer from a country. His name is Gabriel Gulture and he gave us As a requested for the client he gave us a very Specific and very complete spec of what they expected it to be in the back end and sometimes when you work with Drupal We're tight and you say okay We're not going to do these crazy things on the back end because we know it's complex And we know it's complicated and we know it's going to be hard and it's going to take a lot of time But since we got those specs very This thing is full surprises. I'm sorry about that So Like I was saying we got this very specified spec and That was really hard and and we thought it wasn't going to be able to we weren't go going to be able to pull it off because of the time but That's where Drupal come in and it helps you speed up a lot of things On the back end side, we have a lot of back end customization Not only in the visuals, but also in the behavior of the back end Here's a screenshot of what the back end looks like so you know that this isn't a regular website On the back end actually all the the boxes on the right Have information then that can be edited in line and it refreshes with J. Jax That's like a very fancy way of saying that you can edit and save and you don't have to reload the page So that's one of the things that we did there Here are some examples of that as you can see we can just click on the name change it Save and that's it so getting that to work with Drupal You know, it's not easy, but we have to find a way and those were the Simple interactions that we can create to make a more compelling experience and also make this site easier to use Here's another one for input and If you don't want to save or you want to save You get a dialog box where you get feedback on what's going on So you don't have to do something that you don't want to do and in case you're having an issue or an error Then you can easily change it and there's also an undo functionality So if you want to delete something from the back end, you can always have the opportunity to go back in time and say This is not supposed to happen Now that we've seen the video I wanted to show you a little couple of screenshots that may show you how the site works so Basically the site is a map That of course the country that is showing you decide which country it is on the back end and the whole angular app just shows you The map center on the app and you get Very nice interface where you can choose and filter The points on the back that you're that you want to see and we also have News and we also have the latest reports on the right and you can interact with those and those spots will show you what the information of the users that they have Input some information in there so you can see all the information of those reports and see where the contraband is taking place They choose the picture. They choose the tags. They should they choose the brands are being trafficked and They create this little nice block of information that you can consult right there on the website and as you see There are clusters of those points on the map where you can group them together And that that is something really nice that Google does for us in this case in the maps And this site is also fully and completely responsive and it's optimized to be doing stuff Mobile first which you know that what that means that is definitely speed first So that's one of the things that we're trying to accomplish here since this website can be Cloned because basically you can't create a new instance of the site and create a new branded website This is already going on in some other countries right now. There is a project in Guatemala Which is called commercial legal which is using right now the same code base for the front end and the back end And we're currently talking with the authorities in Colombia so we can put the app there through amcham who is doing all the Connections in there and we were also Speaking with other countries in South America that are very interested and one of the countries in in America They're very interested and that it's very interesting for us to see how it behaves. It's Mexico. So we're planning to Deploying the website there and make some more complex things like Getting to know what the performance going to be like when there are thousands of people inputting continents off. So This is one of the coolest things that this website has so Since this is the first project that it's a headless that we have Worked on there are some considerations Of course, we have read a lot about that and there are a lot of articles floating around with information There are certain stuff that we learned that we hadn't read about one of them is Definitely speed When you have an application like this you need speed Not only on the front end and in the back end, but you have to be able to deliver it quickly So you have to think of speed like everything in the project. So One of the things is that the workflow for the coupled front end is very refreshing. It's like Going into a pool on a very hot day, you know, it's like you can do whatever you want You can have a vacation, but of course it has certain limitations but Like I said before when you go mobile first you go speed first So this website if you look at it on the on the mobile phone You get data download of only 642 kilobytes, which is very very fast And it helps in a country like ours and in Latin America that broadband speeds are not that high So we can we can be able to deliver this to anonymous users in In very far away places of our countries and they will still be able to interact in a very well manner There's also something that has been talking about Recently you can have more flexibility on the team members on each project. You can rely on non Drupal developers Which helps a lot because as you know We are scars right good and very good Drupal developers are scars There are a lot of us But we're already working on stuff and we're already hired So when you have a project like this, you have the opportunity to rely on non Drupal developers, which is a good thing for business Also, you can create a very powerful distribution platform. We created an API Which basically lets you publish anywhere, right? You have one API that if mChan wants to deliver a mobile app in the future They just have to create the app the infrastructure and they can just pull our services and show the information that all the users Have created so that is something really interesting So I'm gonna head up and give my friend the rest the work right now So he can speak of all the fun that he had Doing this thing in the front Okay, well, yeah, and this project was actually really interesting Especially for us the members of the frontend team because we were able to use a lot of technologies What that we are not used to go with we are a Drupal shop We do most of our projects in Drupal So we had a chance to test the water for a lot of other technologies There are like that whole thing right now in the in the market We were able to do to use things like young man like grand like lips as we use a lot of of these libraries on these tools But usually we are limited to the all the constraints of a Drupal team or a Drupal development platform So in this project in particular we were able to Use them in a more flexible and more powerful way because we weren't cost constrained to them to the usual Drupal development workflow and one of the Most important features and of this project was to create a single code base a single installation that allows to create multiple Instances multiple sites like like Alberto told you earlier. We have a Costa Rica site We have a Guatemala site and we are planning to expand to other countries So we wanted to be able to manage that using a single installation not to having to copy different Installation of the sites for each one So we have a site that it's able to build itself from all the parameters set in the backup in the back end Sorry, we use angular a we have a single angular installation and that installation communicates with the back end. So The user the administrators go to the back end They can create a all the different instance the instances different sites They specify all the branding information like the logo the colors the categories that users can create reports and the news the all the copies in the different parts of the site and that The front end is able to communicate with the back end the back end sends all the parameters necessary to construct to to build the site in a single JSON a piece of code and And the front end is able to go from a empty template To a full featured site containing all the branding information all the reports all their Article or the news article and all the necessary content and we can have of course different Instances all using the same code base both in the back end and in the front end Using angular using at the coupled site also allows us to work a lot of enhancement in the user interactions in the in the way The user can interact and work with the front-end site We are using a lot of different components and libraries a from angular for instance, we are using ng animate, which is an angular library to manage all the animations and all that These little pieces of of interactions and in a really flexible and efficient way For instance, this this library uses CSS street translation, which which work really good on mobile platforms We're also using a component called an angular Google Maps For to manage all the map relates related to stuff like the map itself the markers the clustering all the geolocalizations and the report windows which show all the information in that in a specific report even little components like the Date and date and time information and are being managed in the browser site. We are using a library named angular moment and That library allows to display all the time and date information in a lot of flexible and customizable a Different formats and we support for internal civilization, which is really important in you know site like this We have some nice features like this one this search without the complete and highlighting which is provided by a library called a and you complete And the site is completely Translatable the said administrator can go to the back end and set the language in a couple of clicks and the angular a Frontend will immediately react to to that setting right now. We have support for Spanish and English, but it's really easy to to add support for for new languages just a couple of settings and adding the translation strings and it will be ready to work in whatever language the Administrator chooses for the site okay as Drupal teamers as Drupal developers We are used to work with jQuery for for most of the JavaScript interactions. We have been over sites, but In this project, we were able to really throw away jQuery and work exclusively without with all the features provided by angular Which was refreshing which was Really nice because jQuery is powerful It's not good and it works really well, but sometimes you it's good to just say goodbye to Drupal behaviors And and work with a different completely different path platform with a much more flexible and modern platform So it was something that our front-end developers really appreciated to be able to to just scrap jQuery and use more modern alternatives We had some challenges for instance social sharing as you might know if you work with single page application Features like CEO and social sharing can be difficult because but Don't get along really well with single page application because of all the JavaScript needed to render the final content so we had to to Do some work around seem for being able to provide a social sharing abilities in the site If a normal user access the page that the front-end app It will be sent to the angular application and have all the maps and interactivity and everything but if Web crawler like Facebook accesses the front-end site a little bit of ph code php code we have a In the front page. We'll redirect that But to the Drupal back end to the original note in the Drupal back end which contains all the necessary Meta-information of the open graph tax in order to build the the shareable snippet With the information about a rip or the news article so it allows to be able to share the content and And a Also give the rich experience of a single web application with angular to the regular users It's basically a very small minoreverse proxy but a without the cashier and it allows all All kinds of Browsers like the bots and the normal browsers to access the content And the specific content needed for for the end each user case And Alberto will told you will tell you about some of the lessons we learned developing an application like this okay, so like I said, we hadn't used this technology before and We thought of course that it was be really fun like Andres said so Everybody wants to use new things on the front-end right now. They're really Unvogue But there are some stuff that still it's kind of hard to go through In Drupal crazy Ajax is very crazy You know you have to choose if you want how far you want to go with Ajax And how far are you planning to do interactions with Ajax? It can be really hard. It it takes a lot of time It takes a lot of debugging and it's hard to debug so you have to choose your battles. Well Since we are open source Fans and we love open source We would have loved to have used open street map and we're planning to continue helping the community of open street map with a lot of contributes and Using more of this technology that they have But there's a lot of research that has to be done I would have loved if we could have implemented it But there is stuff that we still don't know about and we have to do a lot of work with that Still with Google Maps adding content in each inside the map and on those boxes. It's really a challenge It's not something trivial. You can see that a lot of websites do it But it's really kind of hard and you have to always do a lot of research on that to use the best way And one thing that was a surprise for us. It's that one of the stakeholders. It's a very Heavy Windows user and when we showed in the app, he was very excited, but the maps didn't work at all Nothing moved. Everyone was sticky clicks were all over the place Well, so Seems like Internet Explorer added their own touch events which are different from the standards, you know They used to do that with everything right So that is something you have to know So we did a lot of stuff to make at least some things work But there's a y-axis or an x-axis that it's not working right now but it's still the the market for Internet Explorer on those devices that are touch are It's very little. So there's you know An opportunity for them to fix it and we don't have to worry about that that much so there is a lot of Articles around actually about two weeks ago lullabot Wrote an article about should you or should you know not go headless, right? this is a very Important question and you have to decide that and you have to read a lot and you have to choose which project Are you going to make it headless? So you should really check those articles out. We have some Suggestions for you if you want a lot of flexibility in your front-end architecture If you want to do a lot of crazy stuff on the back end If you're sure that you can pull it off, then you definitely would have to go headless Otherwise you would have to battle with Drupal steaming layer and that can be a hard battle If you need heavy front-end interaction, you can also do that That's one of the main reasons that you go headless and if you need to have Publishing platform more than just a single website or a single app then you should go headless You should go and create your create your web services research, which modules are available and decide How are you going to implement that? So we have some final thoughts Some things that we also discovered while we were working on this project and This is very important The things that you are used to Will only work on the back end They are not going to show on your front-end if you want note cues if you want This place we if you want panels and everything that is just not going to work, right? That is all in the Drupal side if you want To have that functionality for your front-end then you're going to have to figure it out in Any way you can but that is not going to be available to you through services You're not going to be able to send that and you will have to work for that And it and invest a lot of time also if you have a very complex front-end Your work is going to be complex It's not that you think that of course you're going to have more fun doing it than doing it with Drupal But it's always going to be complex So it's also a lot of analysis has to be made and a lot of research has to be made as well And also this is very very important Performance still needs a plan and Still needs a plan on both ends if you have your back end you still need your cash Working you still need to to find a way to Cash the content and have it available Make sure that you're cashing stuff and that it gets refreshed instantly Otherwise there is no intention of doing that We we used A method in which the cash is getting cleared, but it's not completely getting clear cleared only that Content that is expiring it's getting cleared. So you don't have to make a whole refresh of all the the service just several parts of it and Also, if you're going to work on the front end, you have to still worry about augmentation minification optimization of CSS optimization of HTML and JavaScript you're not going to just get away With not doing that you still have to do a lot of research on that So I Want to know if if any of you have some questions if any of you have some comments If you want to share your horror stories like like I said in the beginning or if you want to share your happy stories Why don't you just ask us? Go ahead sir. You can go into Mike. Thank you And do you know do you guys implement the push functionality on your apps? I'm sorry sir the push functionality for some of your Website will allow users to report some some some event in in the backhand You can if one user reports a something and the other user can get this update immediately Or they have to refresh that page and got the updates. No actually When whenever someone creates a report that gets into the database We're able to send information to the backhand and it will immediately be Available to the other users. That's why you have to really Invest your time and research what you're doing with your caching But currently it's working like that. We have moderation on the backhand so that Administrations can administrators can decide whether it is a true report But it's a fake report, but it's just a joke, but content is always going to be updated Yep, hi, well, I have a little Horror story Do share Okay, I think for this kind of applications you need to Planify not only how are you going to work on with your backhand or front-end? But also if you really your backhand it's going to be necessarily Drooper we have an application that We started to work on it For luckily it wasn't for a client, but it was an experiment for us And we want to try it. How was headless dropper? It didn't work Because in this case some of the We wanted to make something like an internet and Some lots of that stuff that we were going to do Didn't we're using dropper actually is because all stuff were saved in a custom model and As through power handling all the content we we were generating it was being cached in a way It was hard to to handle a bit So we had to think on another way to do it and we are testing how is it going with symphony? but yes, I think it should be also To plan before Starting to work on these things of if Drupal is good for this for backhand Yeah, actually When we got the specs and and this is very important when we got the specs for the backhand interaction We're kind of questioning if Drupal was going to be able to pull it off So one of these got the discussions was in this timeline Can we modify all the backhand and Actually was kind of weird because when we I think that most of the people that works with Drupal when they Start the installation they remove the overlay module Which is something that almost everybody does but we needed that module So we ended it with it using it to create some functionality for Sites creation we needed to create a wizard. So we we decided to go with Using that module implementing it and see how it goes. So that helps us through the process We actually thought that we could maybe Create a headless site for the administrative part But we said well now that's that is just that is just too much, you know, it's gonna be headless footless brainless So you have to choose, you know Yeah, but still I'm happy that we tried this because now we know What's going on and now we know what we need on Drupal 8 and now we know what Drupal 8 can give us And of course it's gonna be a total game changer because Drupal is still king on these backhand situations But it can be you know a real hard experience Yeah Were you using restful endpoints and if yes what modules and things were you using to generate those if you don't mind? Okay, actually we use the services module One of the main reasons it's that well first we have we had already used the services module we had to synchronize a Big database of teachers from a university from an old PHP system into a new Drupal site So that that way it maintained synchronization also by the time The restful module. I think it's a rustful WS module. I think yeah, it didn't have An RC out yet. So we thought well, we better stick with what we know because we don't know all the Crazy things that we don't have to work on in the front. And yeah, do you think you do you have any? Insights on that Yeah funds for some of the services what we used a restful that yeah for created on for Sending the site information and the news and the reports to the front end after for click also for creating The reports and storing them in the back end in some cases. We had to a program some of the endpoints by hand because for instance the service to create the reports has a lot of custom functionality has a File of loading through a multi-part form. So that part was a program by hand But we relied hard on all the functionality given by the services module Is there a website where we could check out some of that? hand-coded stuff or not not really Not really right now Because well the code belongs to to our clients sure, but we we are planning to start a Block soon, so we'll be creating articles about all this this site and other or size we would create cool. Thank you Thank you. Hey, I know you hey So I was wondering it seems like you know with you know contraband reporting and things like that You probably were pretty unpopular with some people Who were bringing these goods in so did what did you have any you know security issues that you ended up having? address either people you know denial of service attacks or people trying to you know game the back end and maybe try to talk to the API or things like that, and how if so how'd you handle it? It's weird because we haven't had anything like that Yeah, actually I was trying to tempt them with the latest tweet come talk to come see our talk with we'll talk about drugs And you know stuff like that But so far the app has been performing Perfect, so seems like everything is cool so far. Yeah, we've had a lot of jokes a lot of ripple jokes There was some some person who reported that love contraband somewhere Yeah, but we just created a moderation interface. So all reports appear and The administrators are able to delete all like spammy or joking reports But besides that we haven't have any issue with with any security issue Hi From a performance standpoint, how many Ajax requests you guys typically make on that page when it loads When it loads? Well, you know I'm saying like is well, I guess I'm assuming every time an Ajax request is made you bootstrap Drupal so for instance, I mean how many times are you? Requesting something from different endpoints is there just one endpoint you send all the data Just one big thing and you get one big thing back or are there a bunch of little requests being made to different endpoints from the page There is several well, not several but different endpoints We have one endpoint for the site information Which is the one with that access when the user enters the site for the first time and it it has all the information related to the branding and Initial list of of reports, but only that that coordinates and And the number on the category to generate the markers the actual report gets loaded only if the user clicks on one specific report and The same happens with the news the news only are Loaded if the users enter the new set the new section So we try to optimize as much as possible in order to to avoid a extra Extra accesses to the to the API just the necessary to show the the relevant information for each of the of the cases There's a yeoman generator called the head Lee, which will bootstrap Headless Drupal instance with an angular front end right out of the box Have you guys? Investigated that at all to compare your solution to We didn't investigate it for this application. I don't know I wasn't in charge of that part We can investigate a bit with all the guys with which we're in charge of this but We decided to use a angular full stack for this application because in the beginning the Drupal back end We started the development of the front end and the back end in parallel So we wanted to to have like a similar for the for the For the back end for the beginning of the front end and development So we created a very small a similar in in Node.js and we We moved to to Drupal when the back end and all the services API was ready But we didn't know about this Maybe I'm sure we will investigate it and thank you for for for the recommendation Usually Illegal markets are a sign of a sick economy and a badly managed government I'm wondering if there's anything on your app that could help get people into a real Job or economy or someplace good, or is it just like a smackdown on on people? Is there a smack up? Well, there is no snack. Oh, yeah, there is no smack up sadly although We are organization it's part of several initiatives for education and educating Children and educating poor teenagers to to know that technology can help them get out of their situation But not directly through this project. The thing is that we're very interested in that and that's what made us be interested in the project because It was a very tight deadline and there was a lot of stuff that we didn't know but since it is directly related to our Interests that it was a very good investment actually We're trying to think of ways to implement all this functionality that we have here in different areas Yeah, like I don't know maybe three eleven services in our local communities because there are nothing like that in our country So we're trying to see if we can improve Lives and improve communities with this and I think it's possible. Yeah, so thank you Thank you very much for your question. Yeah, it was a very good one Yeah, thank you So is there any oh, yeah, go ahead Did you say that all the sites are running on one database or you split them up into like a multi-site kind of No, they're running a single database where you sing organic groups to manage that the different sites To manage that the different content and different reports and articles associated with each of the sites But everything is running on a single Drupal instance and a single Angular installation also Basically because we needed the a single administration for all the different instances So the administrator can There is even the possibility to have one administrator managing multiple sites and we wanted our client wanted to do everything from a single interface So in this particular case, we thought it was developed approach also We're using pantheon on the back end. So we don't have the ability to have multiple or multiple sites, you know also because our My business partner who's here in the front. He's always against those because they are bad for performance So we're trying to move into different things different solutions. Yeah Any other question guys Okay, thank you very much for coming We have Drupal camp Costa Rica in July would love to see you down there There are a lot of companies coming and this is our information. Please send your feedback There is the link down there and please be able to contact us if you want we'll be around and Thank you very much for coming. Have a good Drupal con