 Shank, I am one of those business analysts and I have with me Ravinder Singh who is one of those developers who are going to talk about the technical discovery for a Drupal 8 project. Hello everyone, good evening. My name is Ravinder Singh. I am currently working as a Drupal, Drupal with seasoned technology. Today we are going to share our experience of doing the first D8 project. So, it is called as a date project as it has been called by Swilgala in the pre-note session. So, we got to know about the date project. So, yes, I will quickly talk about the problem statement. This is one of the websites that we have created about six years ago and this project was in Drupal 6. I am sorry, really after six long years this website needed a complete overhaul and we thought why not go for a Drupal 8 migration, but before that we really needed some kind of an analysis. We wanted to make sure that we are not just migrating the website, but we are also redefining it completely and really making it future ready because we want to make sure that it again lasts for six more years. This was the aim that we had really taken up from the front-end perspective. From the left-end you will see that this is the old version of the website and right-end is the newer version of the website. Of course, from the front-end you won't see too many things here, but really we will talk about what really happened in the back-end. Basically a slide about what really happened is I would like to skip that and really move on to the part where we actually started with the convincing of the client. Client was a big publishing media house in India and really we needed them to be convinced that we should go for a Drupal 8 website rather than going for a Drupal 7.1 and the client really wanted us to know why we would want to move to Drupal 8. Here are the reasons that we really talked about. As Shoshan said that the client is a media publishing industry and we know that Drupal 8 has 200 plus new features available. We have analyzed those features and recommended to client that which features are helpful for client business. The first feature that we identified, better-authentic capabilities in Drupal 8. It is the most important feature that every content management system should have. In a D8 it is pretty much cool in the look and feel as well as a clear layout. It provides in-line editing, it provides editing from the blog or it has two-column layout and also the other cool stuff is that it has introduced few new types of feeds like plus, links, phone, email, comments, etc. Under the cool stuff that is very helpful for the deployment as well which is superior configuration management. It helps us to import, export and synchronize the configuration between the environments without moving the code base. Of course this reduced the maintenance cost as well. We don't need very much exporting according on deployment things. We probably can export importance in the files easily. The other two major factors that we talked about and really wanted to convince the client about was that Drupal 8 was something that supported mobile first approach and primarily one of the reasons why we wanted to do that was because in the last five years we have seen a rapid growth in the users who really want to look at the mobile screens. From a mere 0.4 hours of their screen time it has increased 600% which is about 2.8 hours every day that a person looks at a mobile screen. We really wanted to make sure that this website is mobile ready. We wanted to make sure that it is ready for mobile and up also. We also wanted to make sure that the client is ready to introduce interactive content into the website now. The client being the magazine one he has traditionally really been very scared of introducing interactive content and with Drupal 6 and the limitation that it had, he really never had a choice. What we really wanted to do with Drupal 8 was giving him that platform where he could use HTML5 and getting him the audio and the video support. Then they were of course availability of the new tags like header, footer, navigation which really helped in accessibility also. We will talk a little bit more about it too. The cool stuff that has been introduced which we have recommended for the client to expose web services, to expose news content to third parties. Later on the client can use the web services for getting posted content from the third parties as well as posting the content from the client side as well. So now that the client is convinced to be a guinea pig so this is how we analyze analysis and how we approach the analysis. So this is the D6 site so the main important part was the migration. So now in analysis we analyze the content times available versus used. So as Shashank mentioned that this project is just not just the migration that also we defined it completely from the structure as well as architecture as well as design. So here we reduced a good number of content times. We mapped the phase that which one is in necessity because remember that site is 5 years old. So and even that by doing this thing we of course that we rewrite and we shift the amount of content as needed. So contributed modules. So when we had started this project the pill was the pill it was in DAB release. So we were trying to figure out that which all modules needs to be ported and where we could not spend time in customizing everything so like the DFT, QuickTab, Wordform, Views. So these all are not available by the time. So the customer in some of the code was written in D6 and some of them that we okay so we could manage easily like sharing and pages because these were available as a contributed module in Dupal 8. So that so our decision to move to Dupal 8 was based on that there wasn't too much custom code required needed to port. It really became a really good conversation point. We told the client that listen that the custom code that you have got is really very less which means that I don't really have to port it too much to the Dupal 8 version. Some of it is already taken care of by the new content modules in DH and some we can easily write and we easily move on. So that was one of the deciding factors really that we felt that this we can move to DH. So you are asking. So this is the most important part of maintaining the quality of the product. So if we have old URLs and how that this is the content site, news public site so all URL going to be detect on the new URL. So at the time of the migration we keep a track of the old URLs and in new site we map the old URL to new URL with the redags. And we also got a suggestion from the SEO team to improve the URL pattern. So this help to improve the SEO things. Really decisions. I mean as a mid-term summary we would like to call these decisions that we really had to take. Features versus config management was one of the important decisions and we will talk about it a little later because at that time when we were actually using going ahead with the planning we realized that both features and config management were not really stable. Entities versus node queue. I mean node queue is still not available in Drupal 8 on the other hand. Entity queue also doesn't serve all the purposes. So really it was a decision between whether we want to move, want to port node queue or whether we want to hack or add a patch to entity queue and start using it. Custom versus port of module like Ravinder said that we really didn't have too much time on our hand. We had a deadline. We have to deliver the product by March 1st week. We really could not afford that. We go about doing a complete revamp of all the modules. It was more like a time versus value kind of a conflict. And what we then decided was that we'll take help from the community. We'll ask respective people the kind of module that we want, whether they were looking to port it at some point of time or not. And if they were really looking to port it, we'll know whether their timeline is matching our timeline or not. If nothing is working out, we'll perhaps look for a custom solution. And then some better Drupal, of course. Six years ago, we also were learning Drupal 6. Right now we are learning Drupal 8, but we had better Drupal sensibilities. So really using smaller features like user profile instead of a people content type was a sensible decision at that point of time. Yeah, so after deciding what we are going to use here, how we develop the product, the project. So we organize the trainings for the D8 team from the people that were already into the community. So who are active contributors to Drupal 8, and they are having some understanding, some basic concept, and all these things on the plate. So in the training that we covered, D8 basics, module development, theme development, just all the basic as for beginners just. So those who worked in, who has experience in D7, they found that the 8 side building is pretty similar, so it was easier for them. Then we decided that we tried to cover most of the section from the side building section as this was the media news side. So most of the things were achievable via views, blocks, display settings, all these things. So and other thing that the team was very active on the social channels, the Drupal groups and others like that IRC, and they were seeking the help on the Drupal.org forums, also they were actively participating in posting issues, updating issues on Drupal.org. This helped us a lot because we were always at a crisis situation for a lot of issues, and the one place where we looked for information was the community, the IRC channel or the Drupal.org option. We tried using all of them. We wanted to make sure that we were getting our issues resolved as and when we really have them. Okay, so here how we insured the quality. So insuring the quality was not a difficult task in this project because we had all sides as a benchmark which helped us to defining the acceptance criteria. Some of the experiments got changed by the client after communicating with them. So this was not much challenging task. But yes, the function was one of the challenging tasks that we have to make sure that everything has been moved properly or everything, all the URLs are working fine. So Migrate and Drupal Migrate modules are in the code. We have used them. Yeah, so Migrate UI is not available as of now. So as Dries mentioned, we know that hopefully it will be in 8.1 release. We wrote automation script for testing the migration and we checked the old URL are written properly on the new URLs. So approximately it checked 20,000 nodes. Yeah, we also ensured that all the assets like images are migrated along with the content along with the data and meta information into Drupal 8 site. Yeah, so here that how we planned the deployment. So this was one of the challenging tasks that our team faced. So initially we started using Feature because Feature, we were having the experience on Feature. So that's why we were confident that we can deploy the things via the Feature. Even that the Feature is not made for the deployment but most of the development companies are using Feature for deployment. Yeah, so by the time the Feature was in alpha release, we faced some problem with that. So we will talk about those. And then the config management, again that we faced some issues with the config management. Yeah, because it was conflicting with some other module configuration that which were not getting put on one environment to another environment. So that time the decision was difficult to us that how we deploy and how we can. But we decided to use the partial config management partially and the other rest of the things that we have done manually. Because even that we know that manual is not a good practice but by the time we have to meet the deadline, so we have done some work manually. Now CMI is really stable and we would like you guys to actually start using it. Actually it's a better option. We realized after comparing both features in CMI and after now the better version is released, CMI is really stable and you guys can use it. Now I'd like to now wrap it up with some of the summary points that we really had. The things that we should really keep in mind while we are actually going ahead with our first Rupal 8 project. We have to realize that there are a lot of modules that are not available. And really we will have to struggle around that. We'll have to make sure that we are making a judgment call on whether we want to now port a module or not. Right now the expertise in D8 is very low. Everyone is a beginner and the lack of expertise on OOPS concepts make it further a problem. So we have to make sure that we seek as much information, as much help from people who are actually working on it. They are not really good enough articles that are available for Drupal developers to learn. Loads of research is required. Whenever there is a problem and this is something that we really face, whenever there was a problem we didn't realize immediately that whether it is our problem, our code that is causing a problem, or is it something that is conflicting with something else. At some point of time we even started shouting at the server guys that guys you are making a problem for us and really we had to hear it back from them. We have to finalize an approach for a lot of features really. Another few things that we really did was the search. Apache Solar just released a very stable version on 27th of Jan by that time. We were really in sweat that perhaps we might not be able to deliver a proper search. We were even prepared to do a Drupal search at that point of time. But thankfully now that a stable version is released we are in a better shape. DFP and the ads, that was another very important aspect in our case. We had to write a proper custom module for the DFP where all some of the basic features that we really wanted have been written down. And we have taken those features from the D7 version actually. The whole thing is another problem not everyone is supporting D8 at the moment, everyone is jittery. Thankfully we had gone ahead with Pantheon and they had really helped us in identifying a lot of issues. They were patient with us whenever we threw a problem at them. They really looked at the logs and then helped us out where the problem could be and we could fix it also. But really this all meant was there was a lot of experimentation that was needed. Thank you. Thanks a lot guys. So at the time of deciding the hosting we decided to use the Pantheon only like by that time the Pantheon just had launched experimental D8 hosting. So it was not a stable even that when we are installing the site on the Pantheon it was showing some error on different site the permission to choose and all these things. So none of other system was supporting it or even that every time the PHP version some other version was getting upgraded. So this was one of the challenging tasks. But now Pantheon was fine for our project. Thank you. Thanks a lot guys. Any questions that you might have? The migrated content you are talking about. Yes that was one of the challenging things primarily because the structure had changed but really that is what we really did. When we were using the migrate module it helped us in defining that structure. There were other aspects like URLs that we need to capture and then the redirection of the old URLs to the new URLs. Those are the aspects that we need to really take care of. But other than that the migrate module really helped us in making sure that the migration worked out. How long? Migrate the complete content. Yeah so the actual content migration took a lot of time because there were a lot of cleanups that we had to do. So for example some of the contents had false paginations available. We made sure that all those was deleted before the content was migrated. So the actual migration took us about three weeks but really if the content was in a better shape perhaps it could have been done in a week or so. So when we had started migration so that time we were doing some experiment on that. So first time when we import the content from the C6 site so everything came up, came in our database. So we saw all the feeds and all the relation entity and all things are coming into each site. But later on that when we had to clean up all these things we analyzed, so in the analysis that we defined what things are going to map in workplace. So by that time that's why we have related a lot of content types. We had to tell you on the lot of taxonomies because which are not getting used anymore. Thanks Kamil. Thanks Kamil. Most of the features like Ramendra also mentioned that we wanted to make sure that the ring is limited to the C8 site building only. There were few places only where some custom coding were required and that is what was easily handled because not really too many complex things came up. So really we could handle it with some piece of custom coding but rest of it was all the C8 site building. I can say that ten percent of the project was custom coding. I hope 80-20 as that comes up. Yeah. Almost I am not understanding the initial architecture and the planning for the new information architecture and the delivery. So this was the technical discovery that we did just to make sure that we were understanding what we were getting into like we mentioned that Drupal 8 was a new beast for us. So we wanted to make sure that we were taking up all our time to understand it and it was about two weeks that we took to do this technical discovery. So the project or the technical discovery? The project has been going on. So we are into the last phase of it. The final UAT is remaining with the client right now. Yeah. So it has taken us about three and a half months. So overall technical discovery and the project itself. Correct. What are the learnings that you came from going on from sudden to late? So site building is really not that difficult. It just doesn't take too much of time and people can be up and running. If they have done some D6 or D7 work they can be up and running. On the other hand if there is custom coding involved it requires a little deeper understanding because the way the modules are now written is a little different. So that requires a little bit of time and depending on the grasp of the person's own custom coding capabilities it could take, I mean it could be anything. Fast levels actually do it very quickly. And the quick system is also new for, was a new for the theming team. So this was one of the challenges by that time. So they took some time for fixing the issues making their syntax clear and achieving all the variables in the theme folder. Yes, at the moment yes. Custom coding is the only option if there is no module available. And really the first factor should be how many of these modules are really available. If it falls in the range of 80 to 90% that they are really available perhaps we can now make a switch. On the other hand if really, and we really evaluated this for another, one of our other projects and we realized that only 40% of the modules were really available and there was a lot of custom coding that we really needed to do. So we decided to not really go ahead with duplicate for that. Absolutely, and that is what we also wanted to talk to the community, figure out whether they were doing it or not and whether we can do it. But really that part also depends on whether we have a deadline to meet or not. Like I said, there was a balance that we need to make. We realized one of the modules, the add modules really was needed to be ported. No one was taking it up, but it was an important feature. We decided to port it ourselves. On the other hand there was some that were being done by other people we took help from. So the important thing that how much, what is your requirement in D8 from that module? What is your expected from that module? So if you have a very specific requirement, probably you can achieve those, just those things in your custom module as of now, because then you have to meet the deadline anyhow. It totally depends on the requirement. Which is 7 to 8 would be easier as I think. That's my personal perception. But I also believe that from the perspective of migration and really I have not personally done the migration, so I might not be the right person. But really what I have heard is that from Drupal 6 to Drupal 8, the migration piece is very easy because this is how the migrate module has been done. Exactly. And that is where the migrate module comes in. It has been very helpful. I mean the module itself has been very helpful in moving all the content from Drupal 6 to Drupal 8. We really didn't face too many problems, of course, like Kamal mentioned. There were some additional scripts that we had to write, but really those were not really as bigger problems as some of the other things. So we really think that this could easily have been managed. And the migrate module is supporting for both Drupal 6 and Drupal 7. Which file? When you might. Okay. Yeah, so from the user's perspective, we didn't move, we didn't have to worry. So one more thing I want to add here, that in D6 site, we had an author content type for adding that whose published what. Whereas the actual thing was someone else was publishing the content on someone else's name. Right? So in that case, we fetched the, we migrated the author content type to users in D8. Yes. You said you were a musical dev version. Did you bump on any critical? Yeah, initially we found some issues with the config management as well. We found some issues with the tweak system. But yeah, so with the help of community, we were able to resolve them later. But by that time, we spent some time like two days, three days, four days, like working on those issues. And even that, we organized the code speed and then we came to know the solution. In D6, it was MD5 just. Right? So it was easier to migrate, but it would be difficult in case of D7 when we have implemented hash or something like, right? Flags, oh, this is a Drupal 6 hash, this is a Drupal 7 hash, this is a Drupal 8 hash. Yeah. It was one of the old ones. It unwraps it, use the old hash, and then we can use it. The algorithm is implemented. Sorry? Algorithm is implemented. Yeah. You have made some configuration changes. You have taken it to live. In the live, client will be, want to change something that is there. For instance, you might want to change the test command. Which will change the feature state or the CMI state. So it's going to have a different set of why my client is there. Whereas in your git repository, it's going to have a slightly different function. So how do we sync this? CMI, okay. So if I remember from the last year that previously the CMI was only based on the YML file, right? So now that it is saving all things into database, with the help of UUID. Yeah. Right? Yeah, but it is not fetching from the file. So whatever you have in database, that will load first. And then if you want to rebuild or reload that, then you have to do it manually from the admin. It will be basically overridden files, right? I mean, once we have made an override, and if it is then... Like how feature does, right? So feature, once you reward, then it doesn't apply the changes from the feature file, right? Accept the module file, right? Yeah. Right? Anyone else guys? No? All right. That's a wrap. We have more confidence now. Certainly. Yeah. Now we have one team, experience team, not a team, but experience team, ready to do a project. Now we'll deep dive. Particularly, like, we are more confident now. So whether we should go to D7 or whether we can move to D8, these are the last two slides. These are the things that we really have to keep this in mind. So one of them is, of course, the modules, the other is the structure, right? I mean, if there is a very complex structure and the amount of... There is a lot of custom code that is being written. Those are the two parameters, two priority items that you will have to take care of. And once these are resolved, then you can easily take a call on whether we should move to D8 or not. And even all the big modules are going to port it into D8. So you should be able to take this on to use the D8 for your project. Solve them. Yeah, anyhow, that we organize code speed, we solve them. We really did that, actually. There were a lot of bugs that we really found. We posted them like we said. We really posted them on D.O. Fixed them and then told them that we have already done the patch. Please go ahead and use this. Yeah, but with... Yes. Yeah, this is fine. But you know that the Drupal committee is very big. If you are developing the IRC, probably you can find out some top level of people around you. And probably you can post your detail, your issue in detail. So if you give the detail description of the issue, probably the maintainer or someone asked that who had used or who has faced a similar kind of issue can reply you easily. Actually, in a couple of cases, we talked to the maintainer directly and figured out what was the idea. Yeah, this is the primary approach that we should follow while using any contributed module to the A site and which has some issues. So the maintainer should know first because maintainer is only the person who knows the whole architecture of the module. But the people around us know much about those modules because the module has a big popularity. Yeah, but it's not maintainer. So I tried to contact many of the people but I didn't get the reply some of them. So after that, I tried myself to see the architecture and see the issue what I want to fix. Right? So I tried on that only. So it took the time but yeah, this is fine. Even that, we can ask people around us that how we resolve those issues. That's correct. You'll be introducing spikes. Yeah, I mean, in Ajay terms, you'll be just introducing too many spikes, unknown spikes that you really not know about when you actually dived into the project. And then at some point of time, you might have to take a call immediately. So yes, it will be very risky when you're using that approach. As for the performance of the site, it's a bit personal. What falls in the server after you said that and it was there like earlier there was a problem in the service if you have to say to the server guys, okay, whatever you are, you guys correct. So was there any new server applied to it or what was the role of it? Not really. 90% of the times it was that some faulty module had a memory leakage and we had to fix it to get the things done. So really it wasn't big. It was just that we were facing a problem. We figured out that something is not happening on our local system but it is happening on the server. Let's hand the server guy. But then we realized that the module was leaking memory so we had to fix that. So when we had started the golden beta had not released and we had started with the only the candidates. So that was one of the reasons why we also failed a lot of problems with the servers. Now that the code is pretty much stable, it's actually doing very well. And I think in a very short span, I think Panken would be ready to start moving projects properly in group late and not just on an experiment basis. We had a lot of fights. We'll talk about it later. We had a lot of fights with the client. Really? We had fights with the client. Exactly. We had to really talk to the client. After we were really stumped and we said we can't really do it in the project. Custom? Views. Views. So we have all the views in D8. So there is a big difference between views. So I can't discuss here the difference between views in D6 and D8. And even that D7 and D8 views has again the difference. So we have some advantages like theme system is not available in D8. So you can't modify, you can't create a TPL for a specific field or specific custom field or anything for that. So this is one difference. So you can go third detail about that. What is the difference? Views difference in different version of Drupal? I'm not wrapping it up now. Any more questions? I remember there was a problem of the edit page problem. Initially what happened was that when we were getting into the site everything runs rosy on the local system. Local systems never have a problem. So when we actually moved to the server and we realized that the edit page when we were clicking on the edit page it just blasted or just gave a white blank page. And we tried looking at different possibilities and like I said initially the first reaction was go to the server guys and tell them boss this is a problem and we need to fix it. We really had to debug and figure out and then at a late stage after four days of really real hard work we realized that the problem was with the migrated content itself if the migration was not done then we had not really migrated for a very fresh installation everything worked fine. On the other hand if we had done the migration then it was a problem. And when we actually tried going into that whole thing what really happened we realized that there was one field that had a broken handler and that was causing all the problems and that was leading to a edit page and that was a situation. Really unnecessarily we wasted about four days just figuring out what really was the problem was. Did you have any other thing? And the problem with the entity queue otherwise also was that we cannot edit an entity queue I mean we have once we have fixed it you can't make a change to it you can't change even the content that it is going to use. So if you want to do something else we have to create another entity queue. So that was another thing that we discovered during when we were doing it and we actually created a patch for it so that we were able to edit the entity queue. Again the best practice when we contributed module in a contributed module this is best practice to make it more more genetic and upload it into the issues queue of that module. So and the plus point here the other thing that you can do that you can paste the issue URL to the IRC the IRC or anything or you can ask maintainer if maintainer doesn't spawn you can even have anyone will reply you on that but it it can take some time but if to make it RTBC that is awesome. Right so the first couple of slides actually talked about specifically the business logic of it and how the how we tried to convince the client of course we could not convince the client about the technical aspects of it those are all ours to bear on the other hand we had to convince the client on the specific specific items that that made sense to them so for example so authoring capability was something that we talked about and told them why Drupal 8 was a better option for them rather than going to Drupal 7 or of course a different platform then the other option was other thing that we talked about was the mobile approach and really Drupal 8 is something that is heavily built up around that concept itself so it was easy for us to sell it to the client specifically like I said when the client was looking to promote his website on mobile primarily rather than on the desktop even in the demos he wanted us to look at the mobile screens rather than desktop he said desktop not done I don't care I want the mobile one ready because I know that the users are going to be on the mobile now it's a magazine website right and people are going to just consume content and they should be able to consume it on the mobile level also so those are really the two major aspects of course web services and the leverage of HTML5 or the other layers that we really gave to the client but these were the two really important aspects that fetched us the client's approval and he very healthy conversation guys thank you thanks a lot