 Hello and welcome to our session building amazing searches with search API My name is Thomas idle. I'm the developer and maintainer of search API and several other related modules Yeah, my name is Marcus Kalkbrenner. I'm the maintainer of the solar Search back end and the solar multilingual search back end for search API so this will be a Site building session most of it will be a live demo of how to sort set up the search API in droplet and Few other modules a quick outline of the next hour So I'll first give a few basics about the search API module How it works and then we'll dive right into the live demo With creating a basic search configuration then creating a search view I'm adding a few add-on modules optional ones for additional functionality like auto complete and facets and then I'll hand over to Marcus to talk about the solar and multilingual solar back ends and Demonstrates they are used to and at the end we'll have some time for Q&A to So the search API module. What is it? Why do we need it? back in the olden days of Drupal 6 and before there was just the core search module and Lots and lots of different modules for adding additional functionality to that for adding for searching other types using other back ends like Apache solar and other proper search engines and these modules These modules and generally didn't work well together So you had to have different modules if you wanted to use to search User profiles whether you want to search it normally or with Apache solar and etc So then for triple seven the plan was to introduce a new search framework into core to provide the basic Implementation of search functionality that other modules can then use and work together on and later that was that was then converted into a search by contract module since Implementing that in core would have been too much at once So yeah, that's the basic idea of the search API. It's a common basis for other search modules You can use it to search any kind of date. You can use it to search with any kind of back end and then combine these to Yeah to have Common functionality and you can use any kind of technology to create this or five months to have stable triple eight version as well So yeah, you can use it Both these versions so a quick summary of the basic building blocks of the module There are two basic types of configuration for the module first is a search index Which is the central piece of configuration and tells the module what you want to search is it for example user profiles is it knows comments and Some more details about What you want to search, but all of this is I'm back and independent And then this in an index uses the search server, which is the second type of configuration for telling it how the searching and indexing Procedures actually work. So this is the back and dependent part Which uses then a database or a patch of solar or elastic search or any kind of search engine storage back end whatever to actually do the indexing and searching and These are server back ends are Provided via separate modules. So you'll need one of those For them and other modules the idea is then that those will always just use the search index and So those At all modules this additional functionality will all also be back and independent So views facets ought to complete this will later see I'm all just used that index for configuration and Switching the server later won't affect those that therefore Yeah, now a quick overview about Parts of the index configuration This consists mainly of three parts data sources, which just tell the index what types of data should be in that should be indexed so like nodes comments users user profiles taxonomy terms products maybe Things like that then fields are the properties on the on those data items that should be searchable So you could can just add things like the no title the user name taxonomy term description Things like that product price as fields and then later use those for full ex-searching Or for sorting for filtering for those things and finally processes or plugins that allow you to modify the search in Many different ways. So it's we'll just steal a few example of what they can do a later on So with that out of the way I will dive right into the Into the live demo So as usual Of course, we first have to enable the modules we're going to use for now. This is just Search API itself, of course and the database search which provides the database back and that said We'll need a separate module for that But the database search back end is already so the database search module is already included in the Search API project. So it's not a separate download Then we go to the comic pages for search API and first we add a server as Said let's add a database server and so database server is a Good name for that back in database. I'm a minimum word length 3 is a good default for that, but yeah Yeah, I'll select that as you as you see fit With that we already have the server now. We need the search index We'll call that default index since it will consist of several types of data so Oh, sorry, is this large enough to read in the back rows? Do you want it's larger? It's good like this Okay, great. Thanks Good to hear and Yeah, we'll use the content data source Which of course you usually want but in our case will also index exonomi terms and users just to Choke as that too now we'd have some additional configuration for those data sources available, but it's usually not necessary We want to use the database server of course and Save that second step is adding the fields we want to index So we go to this at fields form First we add an aggregators field which allows us to collect several fields into one And we use this to have a common label field for all three types of content so we'll add Content title The taxonomy term name and the username into that furthermore Then yeah, we'll just add some more fields like the content author which could be useful for filtering But we'll also add the author's name to the full text data So when the a user types the author name in they'll also get content by that author Of course, you also want to index the body the main field of content Shara is a taxonomy term reference in our case and The publishing status is also useful for filtering For the economy terms we want the description for full text searching and for users. We also want to use a status The content title the taxonomy term name and username already included in the label in the aggregators field so we don't We don't need to add those anymore now. We just provide a better label and machine name for the aggregators field Just to be able to spot them more easily For type we want a full text which means that you can find individual words within this field Which is of course what we want for the label and the boost tells us that this field is more important than normal data on the car on the entity So this means that matches in this field will be more relevant than Anywhere than at other places I thought your author name will also use full text but a lower boost because it's just auxiliary date data and the rest is fine as you see we have different Data types here, but usually default is fine like integer and Boolean I will save those changes and now we just need to pick some processors Those are optional, but a good Good idea to look over them content access app automatically takes case of access checking for the content data source which is Very very handy to have of course We want to show excerpts with the search results. So we use the highlight processor The HTML filter takes care that no HTML tags are indexed which also makes sense because you don't want users to Get matches for HTML tags Ignore case tokenize and transliteration are also very good ideas for the database back end in the case of Transliteration, this is only necessary because we have multilingual content on the side We also have German continent to properly treat the umlauts. It's a good idea to have That processor there and we want to index the hierarchy since the genre field is a hierarchical taxonomy term field This is useful to have here then we could change the order, but that's usually not necessary to We want to Just have the excerpts so we set this to never For the HTML filter we only want those fields Processed which actually contain HTML so the content body and taxonomy term description and for the other fields for the other processors all fields make sense to To enable them on for the tokenize we should have the same minimum word length as for the database back end otherwise, yeah You will get unexpected results of course and yeah, just enable it for those two and Then we just index the content and that already concludes the Definition the configuration of the search index and server now. We just need some way to actually search that data For this we're using a view which I guess almost all of you will be familiar with So at a view we call it search Use the index default index and Create the page for the search We also create a menu link But of course that's the same as for any other kind of view. So Just use what you like Now when we see look into the preview we see we already have some results listed here, but They are not linked, which is of course pretty useless and they're also in both languages at once which Also usually doesn't make sense. So first thing is we just go to the fields here It's actually lucky. Of course as a label is the one that gets automatically added this depends on views. So It's a coincidence. Anyway, we enable this link to link this field to its item Option and then we add a filter on the item language Set it to the interface text language and Then you will see that we only get English results and properly linked to their to their pages Then we also as said want the excerpt So this will mean only when there are full text keywords in that case you'll get an excerpt with like Snippets where the keywords are matched And of course we want keywords So we have to add those as a filter. This is the full text search filter. Just add that Expose it and you get a normal search Search field as you will expect and if you have a minimum keyword length set as we did for database back and end tokenizer We just come to give it here to make use aware of it, too And then maybe add some sorting for this like for the filtering we have all the indexed fields available here so the basic default sort is by relevance so Fields as a result for a more relevant to the search keywords will be all of first But in case where are there are no keywords this would result in the Arbitrary sort so we'll also sort by the label for this as a fallback So we sort a sign sending by label descending by relevance, of course and Then arrange it a rearrange them Sorry Rearrange them so the relevance comes first so in case there are keywords the relevant only the relevance will matter and otherwise We'll be sorted alphabetically and Then we just add a result summary header to see what's going on in the view But that's it we save the view Go to the page and here we already see the listing so this This is just an unfiltered list of all the English content we have as you see okay We can probably can't see that but these are nodes. This is a user and This would be one of the shower taxonomy terms and when we now do a search The bodies are all just Lauren Ipsum. So Mac doble match You can see when you get the results the Results are filtered if I now switch to German We just get German results And different ones of course since the bodies are different If we now Search for something in the title Like adventure there's one result and two Three for adventures. So this was not properly handled with the database back and by default You could use the same effort that but I'm Marcus will later demonstrate that it's easily Solved with solar too But yeah, that's already working search view. Of course we could now We could now go and add more filters at better sorts or better search But like the rendering of the results with the theme for them But instead I want to talk about some of the Available at on modules as said The first is there to be autocomplete The name is I think pretty self-explanatory what it does is when the user starts typing into the search field It will suggest keywords that they could try It's at least that's the basic functionality. There's also It's also very extendable so that you could easily add your own ways of suggestions or life results The default implementation with these suggestions is dependent on the back end And so it has to be supported by the back end, but that's the case for both database and solar. So you'll be fine They on that count in most cases So at least for so land beta database So yeah, let's demonstrate that again. We go first of course and enable that module go to the Search by config again, and now we get an additional tab on the index for autocomplete here It lists all the searches it knows about so in our case just the one search view we created We enable autocomplete for that and save and that's already it We now have additional options available, but the defaults are fine at least for our case and we should have set permission so None items can also use a functionality, but we'll skip that too For this demonstrations and now when we go back to the search page, we'll see we have now this autocomplete tag there and when we input some characters we'll see if we get to suggested keywords leaks and learning in this case and if we go to The German search again, then we'll get German Suggestions might in this case So yeah, that's already search or API autocomplete in action I said this also has some more options. It's currently in a better status So we'll hope we'll be able to Release the stable version the month or two but at least it's already as you see very usable and Yeah So I'm the second module. I want to quickly introduce is the facets module Which Can be used as faceted search to the site if you don't know what faceting is You can see it in the examples cringe it on the right. So you'll probably know it from Sites like Amazon or so if you search for something you can then filter by product type Category maybe color or something like that. So this is a dynamic way of filtering. That's much more user-friendly than just having a huge advanced search form and Yeah, there's lots of options available for that too. It's actually not restricted to the search API it can also be used for core search or you could integrate it with other kinds of searches as well and For the search you buy it also relies on the back end for support. So Like but again solar and database supported so Those are fine Anyways, so yeah back to the demo Again, we first install the module Now here's we just need the facets module The essence. It's not dependent on the search you buy it has its separate user interface We had a facet here select our facet sorts with which is again this the search view we created Now we again have all the fields we've added to the index available for this example We'll use the shower field for the facet and Then here we get a huge bunch of options for facets which I won't go into detail here But yeah, you can just look them through and see which are Applicable or sound like a good idea in your case. We definitely don't want to display the entity labels not the raw item IDs so we activate that and Then also use hierarchies in the shore as said is a hierarchical text on my term field Then we save the facets And then we just have to add the block to so that the Defining the facet just gives us the block to place Place into the first sidebar here shower facets place block and Now when we go back to the search We'll see the new shower a facet block right here on the left Now when we search for something now You'll see that this changes too and then we can just drill into the results so in our case we want just High fantasy we want science fiction and we want biographies Eclectic taste, but yeah, so yeah that Just built us for all those categories as you see it's working brilliantly already even though the module is Technically still in alpha, but I think they'll also I'm looking to make a stable release very soon Yeah, we could now of course add further facets on like the author or the creation date But I think this is this already serves a good example and yeah, I'll now just hand over to Marcus to talk about the solar side of things Yeah, thank you Thomas for Basically setting up the page So solar and And First some words why should we use it? The whole demonstration run on the database back end and this obviously has some limitations and there are reasons to use a different back end The first reason we often here is it's faster We like Thomas already demonstrated we have some missing features on the database back end regarding the support of the characters of the different languages and things like that and We get a lot of yeah more features we can Yeah, provide with a with a different back end Just to highlight some we can do phonetic searches for Words that sound similar There's a good integration for location and proximity searches you could do spell checking and a lot more and Yeah, with the combination of the solar back end and the search API integration to a triple you have a real search engine with which gives you Gives you a lot of yeah more options and We Also directly jump into a demonstration So what I will do now is Migrate the site to the solar back end in real time First enable a module If you look for solar we have two back ends This is the Basic back end the solar search integration But we will directly start with a multilingual solar because the site as you already have noticed consists of two languages and contains movies and movie titles in German and English and As soon as you have multiple languages or if you have a single language which is not English use the multilingual back end So switch to the configuration Of search API. So the first thing we need is a different server. We still have the database server enabled But now we will set up a solar server Due to the fact that I enabled additional modules, you'll see more available back ends here I want to have a multilingual back end In the next step I need to tell Drupal how to connect to the solar server. We Have a separate plug-in system for connectors If you run your own solar server these both are fine for you standard HTTP connection and additional basic auth If you run a solar server At a service provider Asks them for their connector plugins. I know there are some from Acvia platform as how a punty on for example Here I use the standard one In this case configuration pops up, which is not the case if you got it from from a vendor I Just use this as an eternal name here And Since the server is not running and I want to have some features I will tell the system that I plan to use a solar six installation I also check these Boxes here because we want to immediately leverage some advanced features And one thing I would quickly talk about is this drop down here Like the different connectors that could be plugged in you can already tell the system about the domain in which you are searching The default is the generic one out of the box. We provide a scientific one because words need to be treated differently like in this example and There are a lot more and I hope that More domains will be pre-configured in the future here. Okay, so Made a mistake the service already running. Okay So what you got now is It looks a little bit different than the database back end The system or the module automatically detects that we have English and German content therefore we have configurations for solar The module ships with around ten pre-configured languages that should be also increased in the future and using this button for Yeah, you can Just for demonstration It's a little maybe it's too small, but we won't look into the content right now You get all the configuration you need for the solar server for this Drupal instant for all languages that exist there You just have to throw it into your into your solar server So We had some issues with the laptop here, so we had to reboot it right before the talk Okay So back to the No To the search API configuration form what you see now is We have a solar server which is not running What we provide is Within the modules if you want to have a quick start Couldn't connect to this was the reboot just a second like to start. Okay. No doctors up and running Within the modules we provide configurations for Docker so you can simply Start these So I'm a little bit lost now Okay, okay, okay. There's also always happening something strange if you reboot it Okay, I had a different solar server in the background So now I start the pre-configured one from the module Okay, here we are I Reload the page now we see that the solar server I configured for this demo is up and running and what we do now Thomas configured a search index and a few for it with all the facets out of complete all the stuff We simply migrated We added this index and swap it to the solar server We do some more fine-tuning because Previously we enabled some processors to improve the search results We don't use the highlight anymore We don't use this ignore case anymore and the tokenizer and the transliteration because solar itself does it better And now we index the content go back to the site and now this search is already Provided by solar. Let's do the same as Tom's tip before also the auto complete is working and What you see now are search results from solar and snippets from solar with highlighting Now let's pick up some use cases Thomas mentioned before Let's search for these adventures again And what you see now are search results Respecting the native singular and plural handling of this language. So we get the search results for adventure and adventures Let's do something obviously we have no results in the for the German language, but let's search for something else Since I know there's an a movie. Let's do something strange We search for plural of Wonderland and as you see we get a result here and The back end probably handles the plural form and also the umlauts within this form And we receive the right results here Um other or languages different than English have different What we require some special treatment For example very common for German are compound words these very long words where you just attach one now and to another And for example Type O in here This movie Yeah, and German Charlie and the chocolate or the chocolate and fabric as you can see we have a Sorry We have a part match here It detects that this now needs to be separated from from the other one, which is also not Possible with the database back end Just as I mentioned we have a lot of features. We get if we use of solar back end Simple or I want to pick one to demonstrate you the additional features I Switch back to to English Okay, just to have the full or basic search result or one thing I forgot To demonstrate Everything that has been configured before is now it's also working all the facets are delivered by by solar now So let's go back to the configuration To the search API and I'm gonna edit the fields of this index and Since I enabled solar I got more possibilities and one thing I want to demonstrate is the phonetic search So what this basically does is that it? Brings up search results is that that sound similar Because I changed the type of the field So system detects that in re-indexing is required I'll do this now Go back to the site and the search And now I type for something like That's obviously completely wrong But this movie Phoenix sounds the same and that's another powerful feature you get With solar and just to demonstrate the multi-lingual multi-lingual capabilities Well, I do something similar here. For example, we had this movie Maybe one more Yeah, and it works also for the different language And the same yeah And I already Combined it with this. Oh, no in this case not Just again Yeah, here we have Hit yeah So and like I already pointed out just yeah check The search API university you will not the university universal You will find more modules for example all the location stuff. You've got fancy tools where you can Yeah, click around and search for for example, yeah Shops nearby or things like that. We should know that you Please ask to reiterate this Session as all the other sessions you wish with it So are there any questions? Are you gonna put the demo up on? Sorry, are you gonna add a video of the demo or a slide deck? Well, yeah, the slide deck will definitely be added to the session and the video Yeah, they record everything here at the conference. So the video will be up as normal by the conference Are there any more questions and Drupal 6 we had something like fuzzy search was was gone in Drupal 7 But I saw an option search for part of a word. So it's working again Yeah, so the database back and does also support partial matching and the solar back and Well, you can just edit the console configuration to enable partial matching for that too or there's the full text angram Field type which also does the same so partial matching is supported by both these back ends Thank you. Are there any plans to Incorporate the logging of queries and the results into search API something which was in search API devil in Drupal 7 I Just so I tweet a few days ago of some module that I think does that and Otherwise will surely add something like that. So I separate module for logging queries and results Before everyone leaves I think we will sprinting here on Friday and Since I'm maintaining the multilingual module what we need is help from native speakers because we have configurations for different languages, but I Can't judge in all cases if the results are correct or and therefore We are here and it would be nice if someone shows up and Especially who speaks the native language. We don't support right now Just to double-check some stuff for example, we have default configurations in the queue for I think a Serbian Arabic and even mandarin so and Any help and this point is welcome Okay, I have a question So Pantheon and Acquia provide solar search And you know we use it on Pantheon and Acquia and for local setup It always choke, you know, it cannot find because it's not set up You know, what do you guys recommend for it? Should we try to set it up on local so that it works? Or you know, what is your recommendation? I assume that it's Caused by the connectors. So you have to switch the connector for the local setup, probably You want to connect the the live index at these providers or your local Development index just local. Yeah, then you should swap the the connector like I showed we have You have to use the standard locally if you have your local solar because what they add is some Authentication stuff and they have their own API and And everything in this configuration if you just pick it from the live site is Pre-configured to connect their server. Yeah But maybe If you come to me afterwards, I can show you directly on the laptop what I mean Okay, then thanks everyone for attending