 Hello, and welcome to our session. I'm Thomas Seidel, and this is Yuri Garazimov. And in the next three-quarters an hour, we'll talk about the new search modules for Turbo 7, Search API, and the FEST API. A quick note at the beginning, this is aimed at site builders who are new to these modules, haven't done much with them. So if you're an advanced user or a developer, this probably won't be very interesting to you. And you could maybe go to the both of our FESA sessions tomorrow at 3.15 for the Search API. Now that this is off the way, let's get started. The Search API is a module I'll start in 2010. At that time, there was a group.group.org discussion about how we should proceed for Drupal 8 core search, what features it should have, how it should be a flexible framework and not just a product like it is now. And I took all of these suggestions and incorporated as many as possible of them into the Search API module in country for Drupal 7. The basic principle was utmost generosity and flexibility. You should be able to index all different kinds of data, use any search engine you want, and display the search in any way you like with as little work as possible. And the structure I came up with, just a quick overview, is the centerpiece of configuration is the index or the indexes. Each index contains information about what data should be indexed, about a specific set of data, for example, all nodes and which fields should be indexed how. And this index then used a server to take care of the actual storage and retrieval of this data. And this index only contains generic information. All other modules built only on the index and thus are completely independent of the server and you could later therefore add another server and use that for the index with everything keeping the same. So now this is explained. Let's get started with the tutorial part. I'll explain how you can easily set up a search for your site. The basic things you'll need are a server and the index and some means of displaying the search. A few pages are similar. And of course you can then add advanced modules also. So once you've installed the search API module, this is what its configuration admin screen will look like. We already have one index defined by default, the default node index, which contains sensible settings for indexing nodes. So now we also need a server. So let's create one. We click, of course, Add Server and for some name and description. And the only important setting here is the service class, which also cannot be changed later. So what is service class? This is the representation of what takes actually care of storing the index data and later searching it. So it can either use a connection to a solar server or store our data in a database or use a sapient server or whatever. None of these is actually included in the core search API project, so this is another module you will have to download and all known service classes are linked on the search API project page. So in our case, I have downloaded and installed two modules, the database service and the solo service module. And as the form is pretty self-explanatory, we use the solo server for this example. So once we select this, we'll get another form for entering the solo server data. Here, we have two options for actually getting a solo server. Either we can install one locally on our own server, instructions to do so are in the installer text file of the solar module. And if you do this and you're using the example server, the defaults entering the form will all really match, otherwise you have to change the port of the path. Otherwise, the second option you have is to use a web solar roaster. There are already a number of them which support the search API out of the box. And in either case, you'll then have to enter the host port and path information to this solo server here. Then just create the server and you're done. This is what you should see. The server was created and can be reached. This tells you that you have entered the correct server information. And also that it is, of course, running properly. What it doesn't tell you is that it's configured correctly. So either way, whichever variants you use to obtain a solo server, you have to remember to use the right configuration files distributed with the solo module. Now that this is out of the way, we can advance to creating the index. For this example, we'll just edit the default node index because creating an index basically works the same. You just have to select the item type you want to index at the beginning. And so yeah, we click Edit Settings here. And now we can change the server to the server just created. And also we take a look at the index items immediately option, which is something you'll want to consider. If you don't activate this, the effect of this setting is that items will be indexed right away when they are created or edited. Otherwise, they'll only be indexed to run runs. If you don't activate the setting, there might be stale data in the index when you edit nodes. And this might even lead to security issues if you're using the index data as a source for access checks. Also, if you use search API for creating listings of nodes or other items, and users can create some items, they would wonder why they don't appear right away in these listings. On the other hand, of course, indexing items right away leads to performance impact, because indexing batches of items is usually, especially with server servers, more performance than indexing them one at a time. Also, this could lead to long waits when a user creates or edits a node, depending on your indexing workflow. But for now, we assume that we have a small side where this isn't a problem. And just activate this setting. We save. And now, because we have set a server, we can also enable the index, which wasn't possible previously. Now, the second form for configuration are the fields. Here, we have all known properties of a node available for indexing. And we can select which properties we want to index, which data type we want to use for indexing. And there are several available, which are pretty self-explanatory, I think, or at least most of them are. It's very important to know the difference between full-dx fields and string fields. String fields are basically a single token, which isn't parsed in any way, just a machine name or something. For example, content type just has a string as a machine name. Full-dx fields, on the other hand, are parsed and processed and can then later be full-dx-searched. So it will be what you want for user-entered content most of the time. And for full-dx fields, you can also set a boost, which determines the relative relevance of this field compared to others. So the type will be five times more important than the body text. And then another option you have here is to add the fields of related entities. So here we have all entities related to a node available. And we can, for example, add the fields of the author to this form. And now we can also index any author fields we want with each node. So here we, for example, index the author's name and user roles along with each node. We click Save, and we're done here. Finally, we changed the workflow settings. These consist of two types of plugins. On one hand, we have the data alterations, which change what data can be indexed. They can, for example, provide additional fields for the node. They can change the data type of fields. And they can also completely reject items from being indexed. So you could, for example, use it to build an index with nodes only of a specific content type. On the other hand, we have processors, which change how the data is indexed. So they operate on the raw data and making it, for example, lowercase or removing HTML tags. So the indexing workflow then looks as follows on a high level. First, we have just a document, for example, the node with a number of properties. We then execute data alterations on it and have some changed document with additional properties and maybe some changed properties. Then we use the field settings to determine which of these properties should be extracted for indexing. We then have just the raw data of these properties. Then we use the processors to further preprocess these extracted properties. And finally, we have this data over to the server for indexing. So to the workflow settings, there are already a number of each plugins available with the core search API module. One important one here is the node access data alteration. This has the effect of automatically adding node access filters to your search queries, which is especially useful if you're using complex access systems. Otherwise, you can just filter by the published state. Or if you're using search pages where you can't just add a filter. But you should, in any way, keep in mind what I said previously about the index items immediately setting. Because if this isn't enabled and you use the node access data alteration, it would lead to leaks when the latest version of the node isn't yet indexed. And so again, we activate this to be on a safe side. And for the processors, it's just important to remember that when you're using a solar server, you shouldn't use any of the standard preprocessors, because solar already does its own preprocessing and would just get irritated. So we deactivate all of these and save the configuration. And this already concludes the basic configuration of the index. We now just go to the status page and index the content. And we're finished. We now have a fully functional and up-to-date index. Now we just have to create some way for the user to actually use the search. We have two basic ways of doing this. First, there are search views, which is a module distributed with a search API project. And that adds views integration for the search API. So you can create views on the search API indexes using filters, contextual filters, sorting, and the whole range of display options available with views for search API searches. And the second option are search pages, which are a simpler way of displaying searches, just giving you a normal search form. But yeah, these are a simpler way to do this, although less powerful. So as search pages are not that hard, we'll use a search view here and show you how this works. We go, of course, to the views admin page. Add a new view. And as the item type here, we use the name of the index we just created, or edit it in our case. If you use search index, it will just give you a listing of all search indexes. This is a common pitfall. So we use the default node index, edit it. And yeah, this is the normal views UI, of course. For our example, we just use a table for viewing the search results, adding a few fields. And I also added a relationship to the author here to include the author name in this listing. Then we add some filter criteria, which is really the core of the views integration of search API. Because here you have all indexed fields available as filters. And you can, for example, give the user the option to filter by content type, or filter yourself, of course. And you can use an additional filter, the search-full-dx-search-filter, to expose a full-dx-search field to the user, which then can search through several fields once, or even all of them. We apply. Of course, we set both of these as exposed. And we also make sure that this option remains at the default of search keys, because otherwise, entering multiple words won't work as expected. So we now just save the settings and view the page. And this is what you'll get. First, you'll only get the listing of all existing nodes. But then you can enter some keywords. Maybe a content type, and execute the search. And then you get a list of search results sorted by relevance by default. As you can see, the results with Mecto in the title are at the top. And but you can also sort by other fields by click sort. For example, here we sort by the author's name. And the only thing you have to remember regarding sorting is that multi-valid and full-dx fields cannot be used for sorting due to technical restriction. And I will be talking about facets for the search API, and actually facets generally for the search for Drupal. And this is mostly applicable for Apache Solar. So when we are trying to understand what our clients wants in terms of building search pages, we usually get some forms with filters, or nowadays we also getting facets. So facets are the links when we have results of the items. And we have a number of results that will be the number of results if we will click on that link. So it's like we have articles, and we have different authors on these articles. And we would like to show how we can make user to limit the search results by the author. And actually historically it was not like this all the time. And there were the cases like we had usual search form, and then we had nice link to advanced form. And when we click there, we get to the form like this with plenty of fields, different select boxes, any variants you can think of. And the problem was that if we type data there and we click search, well, sometimes you don't know whether you will get results at all or not. And that's very bad in terms of user experience, because if you like trying to find shops of electronics that are located in some specific area that sells some specific telephone, for example, when you're trying to go through all these forms, you can spend really a lot of time trying to get the combination of the fields that will lead you to the item that you're trying to find. So nowadays things really changed a lot. Like you can see how Google works, for example. You can see how other search engines, you just have text field with semantics. And when you get the results, you have links on like left side or right side where you can understand that you can limit your results to some criteria. So you go not from very precise tries to get the item you're looking for, but you start from very general, you start typing like shop, and then you see the items that can limit your search with like CT name, product categories, anything. But you already know exactly that when you will click on this link, when you will limit your search results, you will have them. That's the biggest improvements in terms of search usability. And this is where facets come in. Because Apache Solar is great search engine that works separately from Drupal. We index the nodes, the content there, we get results. But with the results, we also get these facets that are actually variants of the links and the number of results that we will get very nice UI from. And these are like usual results that you can find in a lot of websites nowadays. For example, if you go to drupal.org and you're trying to search something, you can see that in your search results, you have modules, themes, documentations, like on that left side slide. So you understand that if you are looking for the module, you can find like 182 modules with these sources. So for like much better example is on the right side where we are shopping. And there are a lot of different characters of these items. So we're trying to find shoes and also we find different types of shoes, sizes, colors, brands, anything. And you can imagine how huge this form would be if we are getting on the previous experience. Like all brands, all sizes, all colors and different types of shoes can have like different types of criterias at all. And some of them are not applicable to each other. Like if we will think about selling electronics, we can sell laptops and refrigerators, right? And criterias of the laptops like diagonal processor memory is not applicable to refrigerators at all. So it's much better when we use facets in terms of we know exactly that user is looking for laptops. So we do not bother him with all other criterias at all and we get the results exactly for laptops. So this is where facets are really good and this is what I would like to talk to you about how to implement them on your search results. So FacetAPI is separate contrib module that was originally written by Chris Blyakas, I think. This sounds his second name. And what he did, he originally developed it for Apache Solar Module and then it became standard for search API module as well. So now it's very great thing about FacetAPI that it works with different modules so you're not going to have different variations of user interface and when you learn it once, you know that you can implement it on different platforms. So on the one side, you can do a lot of things with user interface and on the other side, you have very nice developer experience if you will try to extend functionality. So let's see what we get out of box. Out of box, we have display widgets. Display widgets, it's how our links, I mean these Facet blocks are displayed. Like on this left sidebar, you can see that one block is the links, like this, I think anonymous or admin. These are links with check boxes. You can choose them in settings of the block and then you have simple links and then you can also have select box. Actually select box is FacetAPI bonus module that extend this functionality. So you can write your own widgets. Maybe you want to have some graphics there or some fancy, I don't know, flash application, whatever you can think of or your client wish to, you can implement on display widget in terms of displaying them. Then you can have your custom sorts of the Facet links. This is very handy thing when you have requirements to display them in not like alphabetic order or you want to have them sorted by count or maybe your client will say that we have special partnership with Samsung company and we want Samsung always at the top and this is how you can do this. You just go to the, you can use sorts, plugins, you can write your own small plugin. It will be like 10 lines of code or you have plenty of them by default. So it's very easy to extend, very nice to use and this functionality is already in FacetAPI itself. And then you have very nice functionality that sounds like dependencies. And the idea for dependencies is when you do a search to Apache Solar you get and you explicitly tell to you that I want facets for like whole this bunch of different types of facets, like author, content type, anything, all attributes. And sometimes it's not reasonable to show all of them or for example, you want to make some money of it. So you say to users, okay, this is like usual search but if you register and pay some money you will have much more possibilities and things like that. So you customize what blocks to show and which not to show. And this is where dependencies come in place. So by default you can have different visibility settings for the roles, for the bundles. Bundles means content types. And like Facet API bonus module also gives you a possibility to have dependent facets. That means like same example with shop of electronics. When you type Samsung you will get a lot of different products of Samsung but when you type, when you get to laptops you can show all these facets about only laptops but you don't want them to be shown when you only restrict to Samsung. So you can specifically type like I want this facet only shown when on other facet value is selected. So in this way you can control that user will not have 20 facet blocks at the same time and he will get them eventually when he narrow his search results. This is very handy thing. And of course you can have filters. Filters on facet means that when you get these links you can also take some off the list. And there are plenty of them by default. Like you can show in the facet block the active link or you can hide it so you can show only some specific items or you can exclude specific items or you can narrow this one. I really like like if you have the facet in a lot of like values like maybe 20 of them you don't want to show all 20 but you want to show only the items that have at least like 10 results. So this already is there. So you just enable the module and get this check box and you get this experience for your users. And another thing that is very good standard for search modules is current search block. What it does is when you narrow your search results you want to keep tracking what actually clicked. So you have the ability to go back and you don't want to, you have, it's like on search results you see what you typed, what facets you activated. And also you can change the order and you can also write your own plugins for that system so we can display them differently depending on what results you get. So you can go wild on that part as well. And after enabling, I think, let's almost do. Now that we've heard this general explanations let's see how this works in our example. We go to for the search API in this case of course. We go to the facets tab of the index. And here we have all index fields available as possible facets. We can now, for example, activate three of these, the content type, the creation date and the author. We just saved the configuration now. Get a reminder to go to the blocks page and actually enable these blocks. We move them to the right sidebar. I also activate the current search block as explained earlier and save these settings. And now our example search looks like this. On the top we have the, of the right column we have the current search block as explained earlier telling us how many results we had and what we searched for. And then we have three blocks with facet filters. And we can now, for example, filter by this author and only get the search results for the search that were created by this author. Another thing we can do now is change these facets to OR facets. We just go to the facet display configuration set screen. Use the links with checkboxes display widget and change the operator to OR. And when we save now, we get the chance to specify more than one author. And now we get results by either of these two users. So this gives us another source of flexibility. So this were facet configuration. One thing to keep in mind for this is that facets currently only work with the database and the solar backend, as far as I'm aware of. And OR facets even only work with solar, although it is in planning and work for the database backend as well. So now that we have added facets, what other simple improvements can we make? I'd like to present two additional modules for that. First, there's the search API autocomplete module. This adds autocompletion for full text, input fields as of course unknown from Google for example. This currently works for both views and pages, and but only works with solar at the moment. Now, once we've activated this module, how would we go about using it? We go to the index autocomplete tab, then we have just a listing of all known searches for this index, in our case just our test view. We activate this and click save, and that's it. Now our search looks like this, and once we enter some letters, we get automatic suggestions of how we want to complete this, whereas it will all return results. And you can also, in the settings for this autocompletion search, select that the number of results for these suggestions will be displayed as well. So this is one simple improvement. Another one is the search API spell check module. As the name suggests, this adds spell check corrections to your searches, as also known from Google, where Google will display did you mean blah, when it thinks you've misspelled some words. This, again, works with both views and pages, and also currently only works with solar. So this is a bit different to a configurer. For search pages, you just go to the search page settings and activate this setting, use spell check. For views, you go to the view, click edit a header or a footer, and there you have the option of adding the spell check header or footer. So once we've added this, let's go to our search page and enter some incorrect word. Of course, now both words are garbage because this is just example data, but in our case, metco would be wrong and mechto would be the right search. So this is another simple improvement for your searches, and there are also other projects giving you simple benefits as well listed on the search by project page or link there. And now I just want to quickly go over one frequently asked question I was asked to explain here, namely a comparison of the two available solar modules or the two popular solar modules, namely search API solar on one hand and the patcher solar search integration module on the other hand. Of course, the search API, as explained, has the advantage of being very flexible. You can easily configure things right down into the details and also have a whole range of data. You can index all entities are supported by default, whereas with a patcher solar module, this is more optimized for standard use case of indexing nodes, which it can do very well. But if you want to index all entities, you have to activate or even write additional modules. And also if you want to further specify the details of how things are indexed or what is indexed, you also have to write code. One option, one advantage of the search API is that you can easily switch backends. Of course, if you want to use solar in any case, this doesn't really benefit, but you still can use the database back-end for testing things first and then later switching to solar. What the patcher solar module does very well is multi-side search, which supports for which currently lacks in the search API, at least out of the box. For this, you'd have to program some things. Search API, on the other hand, has better fuse integration. The patcher solar module has fuse integration as well, but it's not as refined yet. One strong point though for the patcher solar module is that it is available for a triple six. So if you're using that, search API isn't an option in any case. The bottom line is just try out both modules and see what fits best for your needs. And yeah, this about ends our presentation. Just a quick reminder for anyone who might have listened. There's a search by both of fellow session tomorrow at 3.15 in the Chamonix room, which I think is in the basement of the other hotel, Western Grandia. And yeah, thank you very much. Now, are there any questions? Please use the mic and it's getting lined there, I think. Okay, hi. Two very small questions. First, you say that for node entities, we need an extra module. Is it a module that any non-node entities is working with? I mean, I'm creating my own entities. Can I use it? No, if you're using the patcher solar search and the creation module, you'll have to enable one module for each type of entity you want to index aside from nodes. So there's a user search module which allows you to search users and a few other one for taxonomy terms. And if you're using your own entity, you'll also have to write your own patcher solar integration module so you can search through these entities. Okay, so that's clear. And what about open search? Is there anything ready for open search or in the coming months? Sorry? Open search, you know, just compatibility with the search you've got in the browser. Well, you know, if you use Google or Wikipedia, you can use the search form in the browser. Oh, okay. And the open search protocol. I guess this would have to done on a per side base. Because they've been done, things done for Drupal 6, but I didn't see anything match up for Drupal 7. So if you heard about... No, I don't know of any module that does this, sorry. Okay, thanks a lot. The solar search integration module has out of the box highlighting of search words in the results. Yeah, search API for solar also does this. Search API for solar, I haven't looked long enough. Thank you. The results of the autocomplete, can this be come out of a view? No. Okay. This come from the solar server. So using a view to get these results would be something entirely different. You would have to write another module for that. Entirely different views, yes. Is there any possibility to have a search block form like the older search? Yeah, normally in the normal search, the module will have a search block form for your site without a search page. You mean with the different tabs for different searches? No, no, in Drupal, we have a small search form and you can put... Oh, a search block? Yeah, yes. Oh, sorry. Yeah, that's available for the search pages module, by default. And for search views? For views, you can use the block display, but not display, but display? All right, display. But... Yeah, you can use this, but I've run into problems because it kind of doesn't allow you to... You have to write one line of code that tells you it can use exposed filters, although it doesn't use A checks in the block, it's a bit complicated. Okay. Basically, it's not possible out of the box, but very simply. Okay, thanks. I have a question about hierarchical taxonomies and how they're searched and indexed. So the scenario is you have a hierarchical taxonomy based on location, and you have, let's say, Germany as a parent and Munich as a subterm related. So in terms of both full-text search and filtering, what happens when I type in Munich as a term? Will it also display all the results from Germany with either facets or search API? How will that be processed? This depends, why would you use full-text search to search for that, I mean... Because the user's lazy. So they type in Munich and they, maybe the first few results should be about Munich and then the rest should be about Germany. Is that possible? Ah. I can answer. I just had the case quite similar to this one. And actually what we have done when we were indexing, like for your use case, if you want like Munich results first and then Germany like beneath, what you can do, you can index your term of Munich as one field and then you look at the parent, Germany as the second field. And then when you type Munich or you type like Germany, anything, you have both terms in your item. So items when they are, when you type Munich, you can like, you can alter a query to add some results, you can add more terms related to Munich. So when you type Germany, for example, you can alter a query to add some CT highlighting and you can do very different things with Apache Solar on this term. So and actually you can also recognize that Munich is in Germany and do two queries in this case. So you can do a query to get the results of the Munich and then you can do another query to get Germany. But it's really very custom. I mean, you will need to code something or you need to build several views. So, but it's definitely achievable. It's not something that is not possible. In terms of also then the facet API, API, is that functioning? With facet API, there's one thing that facets, they are built on one search. So if you, for example, will go for two source results, I mean doing two queries to Apache Solar, you will still need to choose what query to use for the facets. But if you're talking about just hierarchical facets, then this perfectly works for texonomy terms. So you'll get first only the countries in your example and when you click Germany, it will have a pop-up of all the cities for which there are results. Cool, thanks. And facet API, they also have displaying facets as hierarchical. So you will see items and their levels. It's very convenient. You should really play with it and you will find very nice things there already. Hello, I'm the owner of two modules on Drupal.org, Apache Solar file and Apache Solar user. To be frankly, I'm not satisfied to use Apache Solar or search API. So, but for developers like me, I have to think about that a query is behind Apache Solar and they put a lot of power to support that. So I'm working mainly on Apache Solar. But I want to know, is there any plan in the future? Maybe we can join forces to have a unified to search API in Drupal 8. Do you have any thought about that? No, there are no concrete plans for that. Well, right now I got a, my main issue is about the file entity. To search file entity, I need to pass the document like a word, PDF. I think neither Apache Solar or search API provide a good interface for such functions. Do you have any thought about that? No. Oh, okay. For the Acquia thing, the matter is that they not fully support config files of search API solar module. So the difference that you cannot enable search API, for example, for Acquia hosting, that config files are really tied to Apache Solar module. But there are some, I think Chris developed some things like unifying config files, trying to do that so they can run also search API solar there. But also, I think it's not possible to upload custom configurations. Yeah, you cannot, but no, in hand. Oh, okay, I heard, hey. No, the matter is there are things that trying to unite them, but it's not there yet. And the schema is one issue. Another issue is that they have many developers working on Apache Solar. I feel that their part is more well-maintained and if I got the issue, I'm confident that they fix that very quick. Sorry. It's the matter of community, actually. I mean, yes, there will be one, two developers full-time involved in this project, but community is much more than that. So, anyway, can go other directions and we never know who will, it's not being survived, but it's good sometimes to have competition as well. Thank you. Do search API and fasted API support geo-searching? Oh, wait. Yeah. Search API, there is already a search API location module which at the moment works with patching the solar module, but I plan to work on bringing this into the solar module itself and improving this. So, soon there will be great location searches in search API and now there are location searches in search API, but they already look quite nice, so this is possible. And for fasted API, I don't think there's... Well, fasted API is not functionality for geo-locations, just displaying facets. And for the... Yes. Sorry? Yes, yes, but the matter is using functionality of the solar, there is actually modules for patching solar in sandbox, and I know that Nick worked on the code sprint yesterday on also trying to implement some functionality. So things are moving on both sides in this matter of which one to choose as usual. Thank you. When I define a search index, I have to choose the entity type and also on the view, I have only the chance to choose one index. Is there a possibility to search over two indexes at the same time? Yes, there is the search API multi-index search module which does exactly that. If you have more indexes lying on one solar server, you can search all of them at once, although I don't know how I'm working. This is at the moment, but if you have any issues, just go to the issue queue and I'll fix them. Okay, thank you. I wanted to write. I believe I'm right in saying that the Apache solar search module, which the other one does out of the box, it will return results where it not only highlights the results, but it also does a fuzzy search. So like if you search for house, you'll get housing, even though housing is not, even though house is not a substring of housing because obviously for the I, can you do that in search API, basically? Not out of the box, no. You'd have to modify the configuration files for that. Because obviously you've got the alternative spellings which is basically the same sort of concept, isn't it? Where I'll tell you the alternative spelling, but with the out of the box, the other one actually highlights and returns results straight away on the first go. Um, yeah, no, search API doesn't do this by default at the moment. Right, is there any way to configure it so that it does? Um, not in the UI, but with the configuration files of solar, it's perfectly possible, yeah. Right, okay. Thank you. Okay, then thank you very much.