 Hi everyone, it's an honor to present today Progress and feeds my great module My name is Irene Azaz I came from San Francisco. I am Drupal developer open source evangelist My name is Yuri Recognized as a back-end developer that's in the Netherlands and I'm the feeds module maintainer and I've worked on Feeds for the past six years now In this presentation today, we're gonna cover we're gonna go with overview of Content expert and in content imparts and will compare feeds and migrate frameworks We'll do the demo and then hopefully we'll have enough time for question and answers We're using development tools. So be aware Before we dive into specific content issues, I want to take a bird eye view on the websites all together and Remember put it into context. So we remember why we're doing all this content migrations so the function of the website is support organization and because organization change websites also changed and Content is being generated inside organization and outside organization and you want it to be moved around quickly and efficiently Let's look up what we're moving We're moving Structure data in various formats. It comes from different sources. It comes from files It comes from other websites. Sometimes it come from databases We sometimes want to move all the content one time Which would be a migration for example from the previous website Or we want to keep updating news or courses or some content that is has period of periodic nature We want to give instructions to our site what to do with things that we did previously like do you keep previous? Previously imported items. Do you want to update them? Do you want to remove them? And we want to monitor and make sure that we know what actually happened so Today we're gonna look at two large very typical cases One is migration from old site to the new site. It's more or less one-time thing You have very clearly developed structure and you just want to move things. We're not talking great designs We're talking just pure migrations or upgrade Down the road when you're migrating you won't might want to transform something for example Triple eight uses ISO data date format which has take T in the middle and triple seven does not so there are some Transformation that might need to happen and then the new system Follows the old structure We also have different type of cases where we have existing website for example We'll work I work at Stanford University we have departmental websites and we like to integrate news or courses or some other content and we want to pull it from like central registrar or some other or central news and We have one structure in it in our website and they have different structure. So we have ongoing periodic imports where we need to remap Structure so things are getting into our fields, right? And of course it has to be done by Monday, which by the way was yesterday And here's our tool set in triple eight The huge news is that we have one click upgrade that is based on my great module everyone hoped for this one click upgrade for many years and Some people still say it's just one click at the end. You have to do a lot of clicks before We have wonderful module that I love which is called sheets where I can configure I Can configure experts and then I can figure input and Bring in data and then we have very very powerful my great suit That can work with database with files It is targeted towards developers and we also have an option for custom scripts, which are wonderful and very quick But they're not reusable Today we're going to talk about two major input Frameworks how many people has used my great already? How many people use feeds? So I think that you more or less know what's going on in general in both modules, I'm gonna Quickly go over them and then we're going to talk about why it would be really great to have them working together So my great very powerful The it is for developers. It has very steep learning curve. It supports simple migrations and chain migrations It has some UI I would call it some limited UI where you can see it after you run certain number of drush commands You can through UI see what was happening. What has been imported and you have little limited capability of Rollback or re-import things One of huge limitations is that everything including data source is coded And so if you want to change You can change anything in migration and If through UI on productions, you always have to do it in code you always need developer to change migration and That is a huge limitation in cases when we have Content editors who want to change Something remap field or change the source of migration. And this is where model feeds come in and Yeah, I'm gonna give you a brief look about the fees module for those who don't know the module. It's aims to support non developers to import the content and I've shown you the key features Oh wrong one So it's key features are It provides a mapping UI so you can map source fields to Drupal target fields You with feats tamper you can modify or transform the data and It supports various for a file format you can input from CSV, JSON or XML You can input from various locations like a URL a file upload or a server directory It allows a periodic import so you can import Run an import say every hour or every day this is you that's useful if the The source is not stated and changes and you want to import the updates And most importantly it allows your editors to do it to provide source files and do imports So you don't need to consult the developer would have So I will tell you briefly about how these words first you add a feed type Then and you do specify what type of data you want to create like nodes or users you map source to fields You transform the data if it's needed and And then this part your editor could do is add a feed type supply a data source and run the import So this is how the mapping UI looks like on the left side You specify the sources the second column shows which Drupal target fields the source I'm up to and for some target fields you have additional configuration like for them Body field you can configure which text format you want to input the data in because usually you don't want to Specified it in the source file which which text format Something between you want to have to pre-configured so Let me tell you now something about How feeds and migrates could work together in Drupal it so on the one side you just we have Migrate That's very powerful, but it lacks a good UI And as you see on the other side we have feeds this one does provide to you I But it's a complete separate framework so it just means that Contra modules that want to support both input frameworks They now we have to write and the micro plug-in and the feed spell again, so that's double work So wouldn't it be great if the two systems could somehow be combined together? and that would be win-win solution for everyone because developers only have to wait maintain One import framework as the sidebuilders would gain the power of migrate because migrate can do a lot more than feeds can do and Content management managers can import the data themselves without having to conceal them developer. So that's And that might might cost less so So let me tell you about the feeds migrate module. This is a new module built on top of migrate and it tries to mimic the feats UI It's features are that it's integrated with the migrates Equal systems so so it's top built on top of migrate as I said It says you UI tuned for sidebuilders It works with existing migrations. So if you had to developer coded migrations before With this module you can add them in in UI and it works with other triple distributions as well So I'm going to show you a demo from how it's working today So we have a file here an XML file it has two items and for the simplicity of the demo we only import the titles and It's there are two articles we want to import So first we add the migration group We give it a name doesn't really matter what kind of name then we go to list migrations We add a migration and this is where the feeds migrate UI comes in you can give it a name You select the group You select where you want to fetch the data from in this case you want from a URL this You select how to parse the data in this case XML We specify the root expo which I won't go into detail here now and We specify what type of data to create in this case an article notes Then we go to the migration we go to mapping We add a new mapper And we want to map to note title so we select title from the list Do you specify expo source which also happens to be title and because we don't want to input duplicates We check that it's in a unique field and then We go to feeds migrate and this is what your address would be able to do We should let our migration and this UI is called is subject of change, but now it works like this We specify the URL to input from We check them to make sure it's input our articles it's safe and Then the editor clicks on input and it should import to articles Two articles are successfully created Let's see me in the content list Yes, there they are. There's the article lower on ipsum and the article feeds migrate is ailsome So back to the presentation So let me show you what we just saw The what currently is working for feeds migrate that's adding a migration and the mapping UI and performing an import and Documentation is the work of progress And we currently busy with creating DI for configuring process plugins That's process plugins are the equivalent of Feeds tamper for those who know feeds tamper. We don't know process plugins So I give the word back to Irina Yuri, thank you very very much for all the work and All the coding that you've done and everything However, we still need some help I Want to thank first all the developers that already Contributed and get it from the up from the ground this new module. We have architecture in place We have very clearly defined issues that describe things that can be actually done in two hours six hours of your time and We have regular meetings on Thursdays. So today Around eight of six o'clock Locally, we will have another meetup for those who are on other continents of East Coast If they want to get on slack, we're gonna tell them we have seven more people to join the song Please check it out if you're interested in helping We have a wish list of things that are not started yet if you have expertise in this particular area and Are using feeds or needed for your projects? Please join us and make this module more usable and We have a couple of related core issues where we need help Again, this is a place where We are talking to all Drupalers open source contributors please Try the module. We're talking to all the users of the module. Please try it use it say this is what we need This is what's working You don't always have to contribute by coding you can also contribute by testing by submitting By submitting your What doesn't work for your bugs and just simply spreading the word around so people can Start using this thing I want to say thank you to all the avant-garde developers Well that worked on it years during last year and a half and now we have some time for questions and answers before we have next presentation So we were who does have questions Nobody has questions, maybe I can ask a few questions I've been trying to use the commerce Feeds module and trying to import products, but The products are having some multilingual fields. How can you handle that? Feeds I'm still working on the multilingual support in in the regular feeds module and for feeds migrate I haven't Really looked at how that would work Because now I'm overriding the fields. You're now overriding the fields. Yeah Yeah, there is an issue for language support in Feeds 8 That takes three so just that's not feeds migrate, but Yeah, I still need to dive into that and I'm too busy with Feeds migrate and other feats issues. So it's It's too too much for me to handle all the ones so But it's it's it's a plant So go ahead Niels Feeds migrate module replaces the other feeds module and will there be a moment When the other one then the current feeds module then becomes deprecated What is the schedule or idea about for this? Well if feeds migrate succeeds Then then it will eventually it replaces the old Feeds module, but at the moment all my clients are on the regular fees So as long as they are on that feeds remains supported And I think I will Support it even longer I think the only the only thing I would add to that is that You at one point when you do that that you move stuff over to the feeds because there's a brand in in feeds that it's way easier to for For people that know it then to have to because they so we are now in feeds. We there's no there's no way ahead And they don't know that feeds migrates exist maybe And that came reset me maybe say that that's a microphone and the people at home can hear it too The idea is that when feeds migrate is ready. Maybe what you can do is that you make feeds 4.0 We create a migration pass from 3.0 to 4.0 somebody so migrate migrate you have experience And And that we basically that then you get all the benefits of feeds migrate within regular feeds, which is a very popular module coming out the way from Drupal 5 at least perhaps even before works on feet since 2013 and believe the first water was released in 2009 or something or Not sure Yeah, that's a good idea to just call it feeds version 4 But right now it's We keep it separate for Yes, to have focus in the issue kill you need your own sandbox Yes, so the reason why the reason why we started new module Was to keep it separate from existing working Version of feeds out of the box feeds right now support CSV if you want to extend it to XML or Jason or other formats You need a parsers. It's extensive. It's extendable parsers. So when we started feeds migrate We wanted to make sure that we have very clear indication that is works That is fees you are on top of my great but the hope is that It was very important for me to present branding for feeds Because side builders if you tell them migrate, they're like, I'm not doing it. Just no If you say feeds hop of developers of meh meh feeds now So the idea that things can work together in Drupal is very appealing to my heart. So I think that plan was to have feet feeds migrate Be the like a feeds for everything but right now it doesn't because CSV is not in core and XML and Jason aren't core That's are some of the issues that needs to be resolved And I think I'm going to be closer to Drupal 9 when it all becomes one one feeds module Yeah, so what what Rina said and there is no CSV parser in Drupal core for my great And my great plus my great plus sorry My great plus only supports XML and Jason at the moment Subcsv is not supported yet. I think my feeds migrate. I haven't tried it myself actually So maybe it works. There is a CSV source There is additional module that allows you to so there is my great Migrate source CSV that allows you to In to import CSV sources. There's one more module and She entered to import or something. So people are working Working and trying to make things a little bit easier For site builders. So my great has your eye because it has very strong core But the visual part is a little undeveloped the the discussion with Lucas heading was that Both Jason and XML are supported in core itself and CSV format That's at least my understanding. It's not that the parsers don't support it as well So this is why we said, okay, let's deal with XML and Jason in feeds right now in Feeds migrate right now and live feeds with CSV out of the box as is that So are there any more questions Come here and go to the microphone What was the end goal with feeds migrate? Do you want to have Feeds built on top of migrate API or you want to have a UI for the migrate API? And Because my great is so big we first aim for a feeds like UI built on top of migrate so in the beginning would only support the basic use cases and And we see where we can go from there is not I have not planned that yet. It's first focus on getting it to work So there may be chance that in the future we'll have something more than just a feeds Yeah, yeah, I hope it can go to that So are there any more questions ECK entities in seven are they now migrated to it? Because I tried it at first it didn't work I The ECK is a different model than feed tracing, but there is a there is a migrate pass and I can talk to you all about Okay, as well But it's there So are there any more questions? Okay, then we can finish Thank you everyone for coming and we are ready for the next presenter