 holy cow that's loud all right well we'll get started on time then right about 1045 all right so hopefully you're in the right place hopefully there's a clue that you're in the right place that you can see so welcome this is we're gonna be talking about D8 migration specifically the migrating core modules as well as extending them so that if you have a customer contributor module that manages some data we're going to talk about how do you actually write migrations to get that data from D6 or D7 into D8 so we're gonna talk we'll do a real quick overview of migrating core we'll talk conceptually about how migrations work and then we'll do a real world an actual example all the next three hours so we're in good shape all right so I'm Mike I'm a Drupal trainer and consultant based in Florida you can find me usually in pound Drupal migrate you know most eight to five eight to six sometimes in the evenings East Coast time you're interested and we have Keith I'm Keith DeChant I work for metal toad media in Portland Oregon last fall I launched my first production D8 site including a migration of most of our content from our D7 site which is one of the first times that I've been done in production right wheel I'm based in Montreal I have a small consulting agency and I've been traveling around the world a lot I'm helping people at sprints get started and working on my great so I do a lot of D7 migrates in my day life Ryan has a travel problem so you're taking a break this summer only one city I think all right so this is perhaps the most important slide is I think it's important for us as far as some of the later slides to know who we're talking to so if you do show hands how many people are contrib or custom module developers all right fantastic and then how many people have cracked open D8 and looked at okay fantastic how many people are familiar with the D8 plugins all right few less all right good shape good shape how many people have actually coded in D8 okay there are a few less and then I we're gonna assume there's some people here who just migrate curious who are familiar with the D7 migrate module but haven't really looked into the D8 migrate module or just trying to you know using this as you know kind of that first you know step into D8 migrate you raise your hands again all right good so it's about 50 50 right maybe a little 60 40 and then anybody in the wrong room because now's your chance we won't judge you all right no fantastic all right so let's do a little background let's get everybody up to speed real quick so migrate and the Drupal Drupal data migration modules have been completely rewritten and moved to D8 core so if you're familiar with the migrate and Drupal Drupal data migration modules in D7 what we're gonna be talking about today is you'll see that the these modules are radically different from a code standpoint and from a scope standpoint in D8 but conceptually as you'll see a lot of the same concepts apply so and I'll just give you a little preview if you're familiar with D7 migrate you know you have to set a source you have to set destination you do your field mappings and then maybe you want to massage the data we still have all those same concepts I guess it'd be hard to do a migration if you didn't have those same concepts in some cases we call them different things but they have analogs in D8 the big change and hopefully everyone in this room you know knew this coming in is that D8 migrate modules handle configuration as well as content and that's ginormous that means that we basically can point migration D8 migration at a full D6 site and our destination D8 site can be blank and we're going to migrate content types and fields and field settings and blocks and block settings and you name it it's you know if it's in D6 core cck link email phone might be a couple other modules user and node reference might be a couple other modules if it's in those modules in D6 well we can get into D8 we also bring in the it and modules that were in contrib because those modules are essentially now in core so we reach back into those as well so there's some exceptions that are actually outside of core that you wouldn't expect that are also there right so that's that was a huge change and a lot and frankly a lot of work to you know migrate configuration into into a D8 all right so if you you know clone D8 or download one of the betas and crack open core slash modules you'll see the two migrate modules so Keith you want to talk about the current status right so most of the work for the D6 to D8 migration path is done we're working on mostly bug fixes at this point that is something that we're definitely looking for people to help test if you have a D6 site go ahead and launch a D8 site and give it a try and file bug reports for anything that you find for D7 to D8 migrations that work is just getting started we have some of the plugins written to this point we still have some work to do the good news is that a lot of the plugins that we've already written for the D6 to D8 migrations will be reusable for the D7 D8 migrations if we've and if we've done our jobs right many of the the source plugins for the D6 to D8 migrations will be the ones for the D7 migrations will be similar so we shouldn't have to reinvent the wheel on those as far as the UI for migrate it's not in core yet the core migrate modules as they exist now the primary means for running them is through Drush there are two contribute modules in the works for the UI there's the migrate upgrade module which will contain a basic UI for running migrations and then a more sophisticated UI is in the works in a module called migrate plus which will be similar if you've used the migrate contribute module in D7 the UI for that for the migrate plus D8 module will be similar to what you're used to from the D7 migrate module yeah I think it's fair to say that migrate upgrade module is going to mimic the current like D6 to D7 upgrade page where you just enter in database credentials but the migrate plus as Keith said is the analog of the migrate UI all right and then there are some other advanced features like rollbacks that are not in core yet and statistics and things like that but there's issues for all that stuff and progress being made and then the status of how you can get involved we're going to be sprinting tomorrow and Saturday that can be you know you're welcome to come in and try out test out D6 to D8 migrations or work with us on writing some plugins for D7 we have a IRC channel Drupal migrate that we use for discussing all things migrate and also a Drupal group you can see the URL there it stands for import and there's a weekly call that's listed on that page I believe it's Wednesday evenings in North America 6 p.m. Pacific 9 p.m. Eastern I think it's one UTC yeah so Google hangout welcome to join just to get the status or you know ask questions etc. All right very good so obviously why do we do all this so it can be extensible right obviously old the old core upgrade path was not extensible it either worked for your small brochure site or you were using the migrate module or rebuilding or something else so one of the main points of all this is we need an extensible migrations so the primary means of extending it is with plugins so when we asked before it looks like a lot of people were familiar with plugins so we're not going to dive too deep into it but just you know plugins are in D7 migrate you would write a new migration class plugins are classes as well we're not writing new migration classes we're doing something slightly different but it's they're just D8 plugins so if you understand the plugin system then you should be good to go and the idea is that if you manage a custom or contributed module you should be able to write plugins as part of your module that can handle data that your module manages so as an example if you manage you know module X and when you go and port that module to D8 as part of the code included in your module you might include we would love for you to include actually the necessary migration plugins that allow people to migrate data from previous versions of your module in D6 or D7 to D8 all right so when we talk about migrations there's a lot of different types of data migrations that we can talk about so let's just be kind of very loosely put them into buckets and the easiest one to understand is probably what we call a variable migration if your module is not doing anything more than storing stuff in Drupal core's variable table you can write a migration to map your D6 variables or D7 variables into D8 you can change the name of the variables you can combine variables if you need to but it's kind of the simplest case we have all your module does is you know store variables in D6 or D7 you want to get those variable values into D8 you know relatively easy to write next up on the scale is what we call field migrations and this is if your module you know declares a new field type you know fields are a little bit more complex because you've got field configuration you've got field instance you've got widgets you've got formatters and you have your field data as well so there's a bunch of things that actually would need to be migrated from older versions of Drupal into D8 and then entity type migrations if your module defines new entity type then you know that's a bit more of a complex migration where you have to move you know entity data or entity configuration data as well as entity value data from one to the other and then the other like the everything else bucket if your module defines its own database table and provides crud for your own database table then you'd have to provide a migration to go from whatever your data structure is on D6 or D7 to whatever your new data structure is on D8 so today we're going to talk about field migrations because it's not the easiest it's not the most difficult it's not potentially you know an edge case where you're managing your own data and actually provides a really it gives us a nice opportunity to show how migrations work conceptually and how to extend them a little bit so for field module migration specifically when we want to migrate a field from an example we're going to do today is a D6 field into D8 we actually have to run a bunch of migrations it's not just one migration there's actually six migrations that are involved there's the field migration which is the base field there's the field instance which is relatively usually we don't even have to do anything special for that because that's usually just a reference to the entity that's attached to there's the widget the formatter so those four migrations handle kind of the configuration of the field you know what content types is attached to what the settings are for the field what widget and formatters to use and then there's the values the data that actually gets stored in that field that needs to be migrated as well as well as the revisions the data revisions so for field migrations it's not you know super simple but it's not the most difficult thing we can do luckily the way the migrate module is written and just the nature of some of these migrations for most of the time you really only have to worry about four of these field instances are basically as I said just a reference to an entity and field revision values again are just basically a simple lookup table so unless your field level module is doing something really wacky the base implementation of these is normally fine and you don't have to even we won't even look at either of these moving forward here today all right so let's talk about what a migration configuration looks like and this is kind of you know we were just talking and this is conceptually kind of the biggest shift that people familiar with migrate are gonna have to make where in d7 when you're using the migrate module you create a new migration class so you want to migrate a content type from one place the other that's a new migration class in d8 we have all we basically have one migration class called migrate executable and to create a new migration we pass it a different configuration so as an example in d7 you would have you create a new class and you declare the source the destination the map and all your field mappings well in d8 source destination field mappings that's all configuration so to create a new migration really all we have to do is just extend or write a new yaml file theoretically theoretically we'll say so the work we're gonna do today the example we're gonna show you is not creating a new migration it's creating a class that injects new configuration so I know that's a little conceptually it's hard to understand so I'm gonna show you in a second but the nice thing about this is is that conceptually it's very similar if you've used migrate in d7 we have to define the source we have to define the field mappings and any data massaging in d8 we call that process and we have to define the destination so if you're familiar with migrating d7 these are not foreign concepts these are the direct analog so let's real quick let's take a look at this this configuration file for for d6 field and that's what this is so you can just you can find this in the core migrate dribble folder if you're interested to follow along so I'm not we're gonna go line through line for this so let's just break this file up into groups it's standard you know it's d8 yaml but we basically have the metadata that describes the migration we have our source with our source plugin and similar to a d7 migrate what does the source plugin do it defines the query that gets the data from the source database so that's all this plug-in is doing and then if we go to the very bottom we define where's this data gonna end up in this case it's gonna be field storage config everything else we see in the middle here under process these are field mappings as well as any custom process plugins or any process plugins I should say that allow us to mess with the data so this is this kind of combines field mappings with prepare row and the best way to think about this process is in a pipeline you know on one end we have the source and the data comes in the source real row by row and then we can define different operations on that data as it goes through the pipeline and those operations are defined in here and we're gonna talk about this static map here for a second so this we're gonna go into the weeds just for a minute so bear with me the source provides us some data two of the fields the source provides are the the type of field and the widget type so this is directly from d6 and all this plugin is doing is basically it's a big lookup table it's saying if the field type coming in indeed from d6 is number integer and the widget is number then the d8 field type is gonna be integer so static maps are just very simple lookup tables we have a you know a number of process plugins already built we have them to deduplicate we have them for more special purpose I know blocks have a bunch of them the list are like uppercase and lowercase right so the disease he wants to work with if you want to copy something there's ones to turn things into machine names as well yeah so there's an even a more sophisticated one to reference data imported by another migration right and that's similar to the source migration method in d7 so let me let me let's get out of the weeds here and come back here so for those of you familiar with d7 migrate and this is really important for us because I know for us this is kind of once we once the light bulb came on for us for this this is when we really started understanding what's happening with d8 migrate these configurations are like our old migration classes to find the source to find the destination and map your fields I know in d8 I'm sorry in d7 migrate you also have to kind of define your map so you can track your source ID with your destination ID that's kind of been built in to the source and the source plug and you define your primary key that's done on the destination side so we kind of get the map for free we don't have to define that separately anymore any questions about this part so far because this is key moving forward if you're with me so far we're doing great all right so either either we've explained it brilliantly or we've lost everybody let's go with the former all right so we're gonna dive a little bit just a little bit and kind of review so when we define the source again all this is literally doing is doing a query on our source table and this is what we're looking at here this is a source a d6 content field node table that basically lists all the field types on this you know d6 sample site we have we're gonna come back to this well actually let me let me talk about it right now real quick you see we have a field name field a type active these field names map directly to our process functions so the process functions kind of work just like in d7 or mapping I'm sorry works kind of just like in d7 where it's the destination field and the source field so for this particular migration we're gonna take the source active field and map that to the d8 status field so if this field quote has a value of one on the source then the d8 status for that field is going to be one so this is a very simple field mapping with no data manipulation or massaging whatsoever when we do have to mess with data that's where the process plugins come in so type and widget type these are source fields so type and widget type is somewhere over here source fields coming in if the source field we can even look at this one so type text if the type coming in is text then the lookup table will say okay well here's the type and if the widget type is text field and d6 then the d8 widget or well then the d8 type is just gonna be plain text still good all right fantastic so I don't even think I need to talk about this because I've revisited this a couple times ready this pipeline is super important to you know understand as far as how migrate works this is the right way to do prepare row this is the right way to mess with your data you mess with it while it's in the pipeline and the order of operations whoops is defined strictly by do we have one down here we don't have one down here but it's devines defined strictly by the CML you know this plug-in is going to be run first it's actually run second because there's an implicit one but if we wanted to do something more with the card now cardinality field we just add a new plug-in after this one so if you picture it as a pipeline get the data process it with this process it with this then it gets mapped to the destination field and then finally as in d7 migrate you know defining a new or defining what the what the destination is usually in d7 migrate it's a one-liner you know migrate Drupal 6 no or Drupal 6 destination node I forget exactly what the class is very similar you know we basically just tell it where's this data gonna end up and in this case it's in field storage config everybody good still too fast too slow too loud too soft more music less music okay so the cool thing is is that if you understand the last six or seven minutes then you understand the basics of d8 migrate because the rest of these migrations that we're gonna look at today well we're not even gonna look at any of these other ones barely they work exactly the same the source plug-in gets the data the process section of the configuration maps and messes with the data the destination defines where that data ends up okay so that's the conceptual part that's the big picture source destination mapping now let's apply it so where we on time all right not too bad so we're gonna do is we're gonna take this amazing module called iframe is the iframe maintainer here hopefully not okay cuz it's awesome I mean look at this thing so it provides so this is the module we're actually going to build a migration class for or we've actually already done it we're just gonna review it what does the iframe module do well if you have a d6 site you can create an iframe field that basically loads a url into an iframe spectacular right I know you're with me on it here's how it looks you know you basically give it a title on a url and then we've got some information here about how it should appear so for this example we're gonna move all of the data over but we're not really gonna do anything with the display stuff and we'll talk about why in a few minutes so we're gonna focus on we're gonna move the title in the url well when we move the title in the url that that looks a lot like something familiar right that looks a lot like a link field to us so what we're actually gonna do for this example whoops I'm gonna go to the right slide we're gonna migrate the d6 iframe field configuration into a d8 link field because it feels like the iframe module for d8 should really just utilize the link field type and have a custom formatter where the custom form matter is what actually is where you actually set things like width and height so rather than having a different field type for iframe let's keep it let's just make it a link field type because it kind of looks like a link we'll just give link fields this you know a new formatter option which is to output your link as an iframe everybody on board with that plane all right good we're not gonna demo porting the entire module all we care about is the migration classes and that's that's all we're gonna look at today if anybody wants to take this and run with it and become the maintainer from iframe and d8 have at it it's in the sandbox all right so big picture what are we gonna do we're gonna build migration classes that allow us to migrate from d6 to d8 an iframe field so in my sample site right here we have a d6 site with very little it has an iframe field attached to the story content type keep doing that sorry so as part of our migration the story content type is going to get migrated over the iframe field is going to get migrated over and you're going to see it's you know stupid easy to turn it into a field of type link and then as far as the values we're gonna copy the title in the URL over we actually aren't copying the attributes over but we're just kind of stuffing them into link field attributes not even gonna do anything with them but this is what we're gonna do all right so again I'm gonna hammer home this other concept from earlier we're not creating a new migration our class is not defining a new migration our plug and I should say I'm sorry our plug in its job is to inject configuration into an existing migration so let me see make sure oh actually yeah so it would be possible to just go and export your configuration and change this directly it's kind of worth noting so any of those YAML files that you see in migrate Drupal are the defaults if you were to do drush config export those will dump into your files folder you could change those and then do config import there's a user interface as well if you're familiar with CMI and you could re-import these but that isn't really reusable on a module level if you want to include that in your contrib projects as well you have to also consider that the because the config is all coming in as part of migration and because we're already iterating through each of the fields on the node we want to like catch that wave of activity that's happening there rather than redoing it in D7 this wasn't so much a problem because you would build all of your configuration manually and then you would just come in and run like a very flat migration and just pull stuff into whatever your new entity was but in this case the migration process is actually happening it's like spawning processes and managing the dependencies and dealing with all of that stuff automatically so you don't have to run around so you don't want to do that yeah nobody wants dependency management problems so so working with the system is a lot easier in this case right so just to demonstrate what Ryan's talking about is remember this this plug-in is basically just looking at the entire d6 content node field table that's full of fields and all types and for every row it looks at the field type and I'm just gonna focus on field type and it's gonna look up in this map you know are you a number integer are you a number decimal if it's not in here that field doesn't get migrated so if we have an iframe field iframe has got in here so as Ryan was saying we could do away with one of our migrations just by coming in here and adding adding this technically we're hacking core well we're hacking configuration right so I don't know if that maybe that's not the same as killing kittens maybe it's like you know something less evil than killing kids you're hacking what's it okay I'm not sure that's any better but I haven't I don't haven't got to know you too well so but you can see if we just inject these three these two lines into this configuration then when the field map when the field migration runs it'll find iframe it'll know what to map it to so the class that we write for the field migration this is what it does it just injects this configuration into the into yaml for us it's probably also worth noting that it's not gonna do anything special to this though it's just going to assume that all the fields or all the subfields or whatever are the same so we will need to do some stuff which we'll get but that's in the other migrations yes okay so that's you know for d6 for the field migration that's what our class does it injects that into that configuration so the way we're gonna show it is the right way to do it the way that we kind of design to allow module and can treat well module authors to extend migrate so insert stuff there okay so I was forget what I'm talking about yeah okay so the other migrations work the same just like we need to inject the mapping into the field migration we need to inject some configuration into the field widget and we need to about the field widget and we need to inject some configuration mapping into the field format and we're gonna show how to do that in a second as well field values are a little bit different what we're gonna show for that one is similar to the process row from d7 when the values from the iframe module or iframe field are coming in we want to introduce a new plugin a new process plug-in to mess with them before they get saved out to d8 so I feel that what we're showing here some of the more common things that module authors are gonna have to do all right so everything was plugins up until about a week ago it was a lot harder to do what I'm about to show you we this commit came just in time for this so anyway we can do all of this with plugins and so again we're gonna go down in the weeds just a little bit it's possible to do this and we made it as easy as possible for you to do this we have a base class called specifically for fields other migrations will all work similarly we have an abstract cck field plug-in base class that implements all these methods and for the particular field we're working with we only need to override the ones that have something you know some special sauce and there is a cck field a migrate cck field interface that kind of says you must have all of these methods all right so real quick looking at the interface and we're not going to go one by one through these but you can kind of guess what each of them does you know these allow us to inject configuration into the animals so we have our field we have our field instance we have our widgets we have our four matters and the other methods in the interface the first two here provide the map you know if in d6 the four matter was named this then in d8 the four matter is named this so you can have you can change your four matter machine names on into d8 and you could very easily map them from one to the other you can get rid of one in d8 that you had in d6 you can add more in d8 but the map is what you know kind of maps them from one side to the other and then the process cck field values we're going to talk about this this is where we get to define is probably not the best word declare our new plugin our new pipeline plugin that we get to mess with the actual field data on the way through all right so here's the plan we're going to create one plugin called that we get to subclass our base and then we're going to create a second plugin which is a custom process plugin in order to mess with the data and the nice thing is they're both relatively short and easy to do so everybody cool with kind of the big picture plan what we're doing here so far all right do we believe them all right so all this code I'm about to show you it's in a sandbox Benji who's the big one of the big brains of migrating core is in Australia it's kind of a long trip so that's where the name Benji comes from all right so let's look at this code now I get to use my new trick oh actually you know what I do want to show one thing first before I show that trick I do want to show that this module is relatively simple see this is why I don't want to do okay so I do want to just show here so here is our module the info file will see in a second it's there's nothing migration specific in there it's just a simple standard d8 info file and here is our our sub our plug-in that subclass of cck plug-in base and then here's our process plug-in those of you familiar with d8 plugins know that all we have to do is just put them in the things the psr4 we just put them in the right place and they will be found and utilized when they're needed so there's no hook that we like in migrate for d7 we got to implement hook migrate API and kind of declare them there we don't do any of that with plugins plugins are you know the bomb all we have to do is just put them in the right place and we're good to go so if you're interested in creating these you can always go into the migrate jubile folder look at what the folder structure is and just copy one that's close to what you want but if you're using plugins for the first time make sure to update the comments because the comments are annotations and those actually get processed and read if you don't update the comment it won't work so fair warning yeah there's there's a presentation anybody go to I think it was Joe's presentation five o'clock yesterday on da plugins yeah so if you were there then this is going to be kind of easy for you he also did a series of blog posts he's with lullaby I think right I think he's with lullaby there's just a great series of blog posts about writing plugins so if you need that background it's out there and it's very good so here's for our custom d8 module for our my iframe migration classes is the info file you know nothing nothing special there alright so let's look into our our migration well I don't say our migration class our plug-in where it injects configuration so as we said before it is going to extend cck field plug-in base we have to give it an ID a plug-in ID so it's iframe and we're gonna spend time with this first one and you can see this that's the whole thing so there's not a whole lot there we're gonna spend time with this first one because this is a little bit confusing but if you get this one then you get the other ones so process field this method allows us to inject configuration into that field or d6 field where did it go? Inject configuration into this file so that one little function is what's gonna do the job of iframe that's its job is to inject this so I'm gonna leave that there for a second because we're gonna come right back to it so what we're gonna do is we're going to and this merge process of property we're gonna merge this thing which we're gonna talk about in a second into the type configuration so this is the name of the destination field type so if we come back real quick oops wrong one first day with the new trick we're gonna merge that configuration into the type process all right so what are we gonna inject inject into it we're gonna inject to the map our plug-in ID is iframe so map iframe iframe link that's the same thing as twice I didn't know come on that's the same thing as map iframe iframe link so it's this array structure that when you're looking at this file this makes no sense until you look at the configuration that makes sense all right so this these two lines are doing the job of injecting that configuration for us so process field works on d6 d6 field process field widget obviously works for our field migration and I'm not going to go into detail in this one but it's basically the same thing it's going to inject into the options type process something that looks like type map iframe link default so we're gonna make the default widget in d8 map to the link default widget because there is no iframe default in d8 because we're moving it over to the link field there's a third one that we actually don't even need to implement for process field for matter because that's very generic looking because that just takes a map and does our mapping so we have to give it the map and that's what this get field for matter map does this basically says if the for matter on the d6 side is default make the for matter link on the left side I'm sorry on the d8 side if it's iframe only in d6 set it to link in d8 so if one of you were so lucky to be able to create a custom formatter for this module you would probably want to map some of these not to link but to your custom formatter machine name all right so these three methods are all we have to do to migrate the field configuration for iframe fields from d6 to d8 this first one is think of this as field base we're just saying hey from now on when we do this migration iframe fields guess what you're now link fields field instances are just references so we can unless you're doing something really crazy we can use the default implementation in the base class widgets hey guess what when you are a an iframe widget in d6 you're now a link default widget in d8 for matters you know we have a little map yeah and those are the four those are the four on the configuration side so this for those of you familiar with d7 migrate this is all new stuff because d7 migrate doesn't migrate this stuff right d7 migrates only content this is all of our configuration I've seen some people do this but they have to write it themselves usually and people don't share for some reason alright so imagine there's a line in the sand right on this line everything above is configuration everything below now we're gonna deal with the values you know in our case as an example we're gonna deal with you know how do we get this data from d6 to d8 all right so in this case we need to do a little bit of data massaging from d6 to d8 if we didn't if the fields were mapped directly and there was no data massaging involved we wouldn't need a process plugin it'd be kind of like that active to status in this case we do have to mess with the data a little bit so we want to inject a new process in that pipeline so this is how we declare the process to the configuration we basically say hey I'm gonna you know there's gonna be a new process plugin called iframe field and that that iframe field plugin is gonna get three pieces of data it's gonna get the URL it's gonna get the title it's gonna get the attributes and then all this guy does is basically puts it in the pipeline it puts our process function which I'm gonna show you in a second into the process pipeline for iframe fields so this is the new prepare row in this case we're gonna declare plugin here and this is what the plugin looks like so we have a base implementation for our process plugin we're just gonna extend it this is one of those comments that Ryan was talking about the annotation this is our ID iframe field this goes directly back to this ID right here so by giving the annotation the other one and matching up here that's how it knows which one to run so we we basically have two plugins now if we go back to the file structure this is the second file that you saw there right this is our process plugin now and the process plugin normally has one method called transform think of this as your new process row and transform literally takes data in on one end in the form of a value array and returns you know it on the other end so for the next plugin or in this case after it returns here the next step is to save it into entity field storage so in this case we're just gonna list out the array into URL title attributes over bit we're just gonna map it so the source value becomes uri and d8 the title to title value and here's the sum total of our massaging we basically just have to unsurrealize attributes so we can do a lot more stuff in here in this case it's very simple but you know what we're focused on for this example is the structure rather than the details of how we're actually going to massage the data makes sense good everybody excited about migrate there's only you know a few of us here tomorrow so be patient with us because I know you're gonna be sprinting tomorrow now because this is you know it's cool stuff all right um yeah so I guess let's see an action right I guess that's the next up there's not a whole lot more there's really nothing else to it as Ryan was saying let me just go back and reinforce what he said there here's the process plug in plug in migrate process here's our I'm gonna call it configuration injection plug in because I'm not gonna call it a migration plug in despite the title of the session ignore that you know nothing all right so it's you know this is for a real field migration and it's what a hundred lines of code maybe you know take away the comments and the spaces and it's very compact so we think and I'll say we think Benji did his job because Benji's one who designed all this we just kind of eat it all up and use it all right so let's see an action so again as a reminder here is our d6 site you know we've got a couple of nodes with i-frame field we're gonna focus on this guy right here here is our d8 site I'm showing you this because on the d6 side this is a story content type story that's where our i-frame field lives so after the migration we're gonna have a new content type kind of like that one slide I showed you where we're going from d6 to d8 we're gonna migrate the content type and that new story content type is gonna have an i-frame field and there's gonna be a new node on the d8 side that is identical to this node right here but it's obviously it's back and display it as an i-frame because we haven't written a custom display for a matter it's gonna display it as a link field on d8 that's our social contract that's what that's what I'm promising is gonna happen so live demo so Ryan told me not to do this I'm doing it anyway I haven't changed anything on this system hurry on time alright so you know 15 minutes or something goes wrong yeah what could possibly go wrong here alright so what I've done so far you can look at my history here I have I did a fresh site install so the fresh copy of d8 and I enabled migrate Drupal and i-frame and i-frame is our little custom module that I just showed you so I'm going to my history my great manifest I'm not gonna parse this but this is the way it's up big enough for everybody I can make it bigger we have the technology so this is the way the migrate manifest command looks and my great manifest is a little bit funky but it basically we have I think is it 92 migrations or 92 migration configurations in my great Drupal just like I just say around 100 around 100 like I showed you six of them right that little image with the field field instance that was six I think for d6 to d8 we have somewhere around a hundred as Ryan says if you do I'm not config dash list grep migrate it'll show you you'll see like that's pretty much all that comes out of that command so so I'm not going to run all that it'll it doesn't actually take that long but I'm going to run this d6 manifest node how's that if I point like that this is just a YAML file that just lists the migrations we want to run so this list d6 field d6 field instance field format or widgets node all of its dependencies so it's a subset of all 92 there's maybe 20 that probably that's the 20 that we're going to run so we basically tell drush run this manifest you know we're in our Drupal 8 site and we're going to point at the d6 site so one of the other big wins we get from migrate and core is it works like migrate the source database is untouched we're just reading from that so you can run this against the live databases many times as you want so we run this and you can see the migrations that are running here so filter format user role user so these are all because they're dependent on nodes you can't run nodes unless you have users for authors can't run users unless you have user roles there's that little warning at the end that's just a bug so you can see that cck field values page and story the last two that were okay so that's bringing field values for the page content type and field values for story content type over so that last one that's listed hopefully brought over our iframe field data alright so against all odds we now have a story content type with an iframe field that's now a link we come to content we've got two this is the one we're interested in let's look at edit first because this is if something's going to go wrong this where it's going to go wrong and there we go there's our link text and if we look at it it doesn't you know whoever who's did someone already volunteered to write that format was it I thought I saw something but your job is to you know turn this into a beautiful iframe because everybody loves iframes I do have a recommend for this actually there's a module simply called YouTube that works for Drupal 8 and it's a field for matter for text field and it's a good basis for creating a field for matter for the first time with the settings page and other things like that for matters are just plugins yep so it's you know if you want a first contribution to Drupal or just to get involved with something you know this you know iframe module is not going to set the world on fire but it'll give you a lot of experience so let's say and I have to be honest with you I actually think there's a lot of modules since I've been working on this long time I have to stare at D6 sites for a while there's never embedded media field for D6 I believe that was just a type link as well with custom formatters that's a great opportunity for a potential module author to work contributor to just write the new the write the formatters for D8 and to be honest the migration classes are not going to be all that different from what we're looking at here so all right do we cover everything ready for questions any gaping holes we got seven minutes for questions all right seven that means to quit so I line up at the mic yeah please if you have a question or the mic and just so some background in three of us there are definitely people in migrating core who are a lot smarter than we are none of the main architects made it to Drupalcon so they sent in the scrubs we are more of them I think it's fair to say we are more on the side of we kind of understand the architecture and we can use it but if you have any high-level architecture questions we might punt if you know the answer and we don't feel free to like put up your hand and run to the mic because that would be useful to get into the recording so please make sure the mic is on great thank you likely be much sooner so the the d6 mappings are in core now the differences between six and seven are not that substantial I would expect that as people get working with it it will proceed pretty swiftly once it's there there was a hold back for a while because it was in a sandbox that was out of date and I believe that's been extracted now and is now under current development so there's a patch available and that's if you want to come tomorrow that's a good opportunity we need I know that we know there's a lot of people who are kind of itching to help with the d8 to the d7 and d8 stuff and we just haven't been ready we are on the verge of being ready and so we can get that patch reviewed and get that first patch in that's kind of the first domino I would expect over the next few months yeah so our goal has always been for 8.0 d6 to d8 migration because d6 is being sunset and that's the most important one so we really have blinders on and then really refers to the UI more than anything because stuff's already in core so I would expect the d7 to d8 stuff is going to be a fraction of the effort the d6 the d8 stuff was so yeah so you're asking if we need for a given field migration if we need one plugin per thing we want to mess with is that what you're asking no because you get in in for a given migration in the transform method you get the data as it is in the pipeline so that field has no 13 subfield so to speak or 13 field values you get access to all 13 of them so you can mess with all of them in that one place and I do we want to mention I mean there is you can still do prepare row in d8 we just what we showed you today is kind of what we feel is the best way to do it it's kind of the there's a air quotes the right way to do it yeah there is I think it's a migrate dot API dot PHP file that contains an implementation of prepare row and you can use that if you just want to use like a dot module file and do your d7 strategy of like dirty hacks mom will be impressed so yes yes to number two that we have that was my great extras in Drupal 7 kind of had stuff like that so I would imagine something like that will emerge in well but that's not considered best practice best practice is your module the data that your module is responsible for should be in your module the migration configuration for that module should be in your module and that's one of the yes there would have to be right but like you said that module could just be very similar what we just showed you is it's putting it into a text field or something it may be worth checking in to see if it's a widely supported module there's been talked lately about putting some of these plugins into core modules as opposed to having them in migrate that's a discussion that's happening now so it may be worth checking in with some someone in the migrate Drupal room or something like that in IRC when you come to that point so just as a quick addendum to that is if I can just pull this up real quick so you can see right now in core let me get rid of this because that's not good still in migrate core we have all of these field types that are not plugins yet this is tomorrow basically if you want to come help us out tomorrow if you followed what we did today you can help us you know write migration configuration plugins for all of these because our goal is this you know this map should be empty for core all of these should be implemented as plugins rather than this map so just you know put your shades down lock the door don't let anybody see it but we all I mean we all you know depending on the budget of the client you know sometimes that's what you have to do you know the the goal of this session was mainly to show there is a right way a supported way of doing it but as we showed you you can also shortcut it just by messing with your configuration files you're welcome it's a wasteland I go into Drupal.org slash project slash Drupal into the issue queue filter it down by migration system Ryan IRC that's the answer well I I'm often not in IRC if I'm not a Drupal cons or events so but go into the migrate Drupal room and just ask in general and someone in there will probably have an idea there seems to be people around the clock so there's a patch that's in the queue that is like I said the first domino and if you ping me or or Kate a Chan or Benjy or anyone we will point you to that issue and depending on the current status of you we status of it we might have you you know test it or extend it or something but once that domino falls then we've got you can look in the issue queue right now there's a lot of kind of almost empty issues they're not empty they have a title about let's write a configuration for this write a configuration for this all of our very our missing variable migrations all right see tomorrow maybe all right yes it does and revisions the revisions come through by default because the promise has always been to bring data from your old system so there's actually a couple little bugs that are worth noting if you at one point enabled the book module and then disabled it and you didn't bother enabling it in d8 the migration system at this point wants to grab that data because you have data there so just a fair warning that if you have a module that you install at one point that you don't use anymore you may wish to consider to