 Good afternoon, everybody. I think we're gonna get the session started here. Thank you for coming. Welcome to our session today on improving your search with Search API and Solar. Big thank you to Nashville for hosting Drupalcon this year. Appreciate it. Just a heads up for everyone that this session is intended to be kind of just to get you going if you've never set up Search API and Solar before or walk through the process. So you kind of know what to expect. It's meant to kind of be a good starting point so you can have an improved search with Drupal instead of just relying on Drupal's core search. I'd also like to ask that you hold your questions to the end. I will leave plenty of time to address questions and we also maybe spin up a little demo site at the end if we need to take a look at anything in depth. So So who are we? I'm Adam Erickson. I'm a senior Drupal engineer with Four Kitchens. Also a hockey fanatic. I coach youth hockey from Minnesota. And hey, I'm Jeff Tomlinson. I'm an architect at Four Kitchens. Consider myself a generalist and a beer geek, so you can help buy me a beer after. We are Four Kitchens. We build websites and apps that help you publish great content across all devices, platforms and experiences. We work mostly with media and publishing and education, nonprofits, anybody with large amounts of data and they need to deliver it to multiple platforms. So yeah, why are we doing this talk? Quick thing first though, I'd like to take survey. How many have had to set up Drupal search before on a website? Okay, good. Quite a few. How many have worked with search API before? Good number. All right. And what about solar? Have you set up solar instances as well? Okay. All right, let's get to know. Well, we're doing this this because search is important and for many users it's the quickest way to find content that they're looking for. If you have a lot of content, you owe it to your users to provide an effective search. Search can be a tool, should not be a crutch. Obviously you want to have a good navigation but obviously you need some kind of fallback there. Another big reason is most site owners want to improve their site search. They want advanced features like filtering. They want more control over their relevance of the information that's displayed to the user. Users also expect better results. We can thank Google for that and obviously most cases we don't have that kind of backing. So this is a good way to get an improved search without a lot of effort and obviously you can't get close to Google, but at least you can do a better job of it. So a quick look at some statistics related to search. On average, 59% of visitors will use internal searches on a site. And on average 50% of people start on a website and go straight for the internal search box. And also on average 15% would rather use search function overall. So what do we make of this? Only 15% would rather use search, but 50 go straight for the search box. On average, most people want to navigate having and they have to use the search. So your navigation architecture is important, of course, but the search is definitely a backbone of the website and how people are going to find your content. So where do we start? It's a very important topic, obviously. Well, what we like to do is get a game plan. With any major feature, you've got to have some kind of outline and know where you're going to go with this. So it's important, they ask questions and do some good requirements gathering. You want to consider common features. These common features we're going to show you today. But the key thing here is don't make it too complex for no reason at all. Some of these common features might not even be worth having in your search. You also want to consider your users. What kind of users you have? Are they just looking for general information? Is there specific information they're looking for? Are they looking for media? Are they shoppers for e-commerce experience? There's many things, obviously, users come to a site and they want to find their information. So you want to definitely keep those things in mind. So let's look at some example questions and things you would maybe want to cover with your team. The big one is what content should be indexed? Consider all nodes that have a full display. That's a pretty obvious one, but sometimes you might want to avoid indexing content that's not going to be useful within a search parameter. Or maybe you want to index entities like media or files. Solar has the ability to do these things. We're not going to cover that kind of thing in our walkthrough, but I encourage you to go and take a look at it because you can build some very powerful searches for media related things or files. You want to talk about multiple indexes. Consider those because if you have a lot of content, a lot of different content types, you might want to break those things apart. That way you can have better performance when you're indexing your content. You can set your cron jobs to index specific indexes related to certain content types, that sort of thing. You also want to identify what content needs priority over others. It's a very important thing to your users to find this stuff that's probably most relevant. And then you also want to take a look. Do you need to filter your results? So in most cases, this is probably true, but you know, some cases you might not have the need or you might not have the structure there to provide faceted searches or anything like that. So, but it's just definitely a question to ask. And then you want to consider different ways of displaying your search results. Most cases it's a common look and feel. They should be pretty consistent. There's probably some rare cases you might need to display something different. Events come to mind. So, normal content, general content, you just see maybe a snippet of information. But an event, you might want to include the event date, which might be a separate field, so you might want to consider how those things might look. So let's look at some solutions that are available within Drupal. Just a quick cover, a few. Obviously our topic today is solar, or I should say search API and solar, but we're going to take a look at a few things here. So the first one's core search. I'm sure everybody knows this is a quick way to get something at least up and running. It's quickly just enable the module and you're off and running. The big thing here is it's limited in capabilities and there's low performance. So if you have a lot of content, you definitely do not want to go this route. You can take advantage of Google and use their search implementation and you do gain a lot of performance by doing this. But you're also at their mercy for their indexing and if your content's updating on a regular basis, if they're not crawling your site, you're not going to see those results. So then we come into search API with the option of using a database, which is certainly a good option. You get all the advantages of using search API's configuration. It's highly customizable. You get better control of your search function and behavior. You've got great community support. And if you have the caveat here though is if you have a lot of traffic and a lot of search use, it's going to have an impact on your site's performance. So you probably want to avoid this option. And that's where search solar comes into play. And so our main topic today, and you gain actually some extras by using solar, some some some additional features that are available within solar that are brought through a lot of good community modules that will give you these things. We're not going to touch on all those details, but definitely encourage you to go and check it out because there's a lot of good stuff out there. You will get better performance when you have a lot of traffic. The one thing here though is it requires more setup and then obviously there's more resources because you would have an external source for hosting your solar application. And one more we want to mention is Elasticsearch, which is newer technology. Again, very high performance. A lot of people are enjoying using this now. It can work with Search API and the one thing here is it's newer technology, so it's not quite as well supported within Drupal, but it's gaining ground fast. And then you can take advantage of some Elasticsearch tools like for analytics monitoring and reporting and that sort of stuff. They got a lot of good stuff built in. So here's a quick comparison chart of the options. You'll notice that we through medium kind of level, just a quick look at what you can do. We put from a support perspective on on solar, we said medium just because you do have that extra little thing. So if you're not familiar with setting up those servers or how to do that stuff, you got to do a little bit more looking around and figure out those things. So but it's definitely, I think it hits the sweet spot for making a quick improvement to your Drupal site. So why Search API and Solar? It's performant. Solar is an application that handles indexes very well. That's what its purpose is. It can run multiple indexes, these and additional, it does require additional resources as we talked about, so you need to have that solar instance hosted somewhere. It's highly configurable. There's an extensive list of community modules out there. It works well with views and facets and we'll cover some of those things in our walkthrough. It's built into major Drupal hosting providers. That's definitely a huge advantage. And it's battle tested. Search API has been around since about 2010. There's over a hundred thousand installs and 30,000 of those installs are using solar backend. So you know it's a good reliable tool to use. So let's talk about what you're going to need in order to build your site or build your search with Drupal search or I should say Search API and Solar. So first thing you're going to need to do, you're going to need to have some kind of instance to host the solar application. Some common things here is popular, obviously, is using a Drupal specific host like Pantheon, PlatformSH, Aquia. They have some great options there. There's also other hosted solar options available. So take a look at those things if you feel you need to branch out a little bit. A few options there are WebSolar, OpenSolar, SearchStacks. Just to name a few, I know there's many others out there. Of course, there's always the option of rolling your own. Self-hosted option. If you've got the ability to maintain that sort of thing, it's definitely not a bad way to go because then you have full control of everything. So for this walkthrough that we're going to go through, here's the things that you're going to need as from a module's perspective is, of course, Search API, which is the core search feature. And then you need the Search API Solar search module, and we will have links to these two, by the way, on our resources at the end. You will need the Facets module. That will give you some advanced filtering. And then the Core Views module. So it's just a just using Core View, it's a real simple setup, so it don't need any additional modules related to views to do what we're going to do today. All right, so before jumping into the walkthrough, let's talk about some important things and things you should probably expect. We're assuming that you already have your Solar instance setup. We're not going to go through the process of setting up a Solar server. So we're going to walk through the setup of the server and index and the steps in order to connect Drupal to your Solar server. And we'll show you how to set up a search index. We're going to talk about a strategy for indexing paragraphs. A lot of people use paragraphs here yet. Yeah, okay, good. So there is a strategy to take advantage of this and make sure that you do get your your content indexed within paragraphs. And we'll touch on a way to get around this. I will also talk about fine-tuning your search results. So using processors, how to prioritize content, that sort of thing. And then creating the search view, which is going to be the display for your users, provide the exposed filter for searching. And the last but not least, we will show the basic setup of a facet. And we're more than happy to talk about some advanced features there at the end if you have any questions as well. So here's an important thing to know. Search API servers with this term is not to be confused with an actual server itself. It's actually a configuration entity within Drupal. These entities are set up to manage the connection to the back end where your data is stored. And these actually could be a database as we previously covered. It could be solar. It could be a LASA search. So you're basically just setting up that configuration. So when we use the term search API servers, that's how it's defined in the admin configuration. Search API indexes, these are also configuration entities that define which content is made available to search, how the content is processed, when it's processed. So you can have it processed immediately or run on a cron task. When you set up an index, the entity types, you're defining what entity types and bundles are indexed. You're also defining the fields within those bundles that get indexed, pre and post processing content. And these are things like access checks, word stemming, keyword highlighting, that sort of thing. There's lots of options available. One more thing to make of note is facets and what are facets? It's basically just a complex structure of filtering that can describe all different aspects of an object. So it's basically just a super powerful filter. So a long time ago, you'd use view search and you'd use exposed filters that way, well facets actually give you a lot more power. The great thing is they have the ability to keep context to the current search results. They have the ability to exclude objects that do not meet certain criteria, and that includes the facet itself or even the options within the facet. The facets are very flexible. So you've got all kinds of different widgets available right out of the gate and you have the power to customize those things. So you've got check boxes, boolean selections, even dependencies. And then you get the advantage of filtering with speed with solar, especially when you need that performance on high traffic websites and you have a lot of searches happening. And then the last piece here is just with views, we are just using the internal views module with nothing extra. The view is based off of the index that we will create. And it's using, we're actually going to be using fields within there, within the views display. Oftentimes nowadays is using view modes and just relying on the view mode of the node to display the information. But we're going to use fields so that we can take advantage of things like keyword highlighting. I'm going to turn it over to Jeff and he's going to go through our walkthrough today. So this walkthrough is going to be, we just, we're doing this with screenshots. Hopefully it's going to speed things along. But as Adam mentioned, we do have a little local demo site spun up that if we need to dig in a little bit deeper and look at some configuration in the questions after we can get that going. So yeah, let's get started. So this walkthrough, it assumes you already have experience installing modules, creating content types, and using the views module. And I already, yeah, we'll look at the demo site afterwards. But first, let's talk about setting up the search API server. As previously mentioned, this is where we configure our connection to the solar backend. So you would go to the main search API configuration page, which is under the configuration menu search API, and click add server. It'll bring you to the next page where you give your index a name and be sure to pick the solar backend. If you have multiple backend options, like you also have database, backend module installed, you'll have the choice of choosing either. Just make sure you're selecting solar here in this case. That's right there. And then these are the primary connection settings. If you're not sure of the values to enter here, check your host's documentation. Also, some hosts like Acquia have specialized connectors available through additional modules that can help with this setup. Choose whether the connection requires authentication or not. Choose your HTTP protocol, your host, the port you need to connect on to your solar backend, the path to the solar core within solar itself, again on the solar server, and then the name of the core that you need to connect to. Here are some things here. These are defaults. You can usually just leave these as is. You can set timeouts. If you need to expand timeouts, you've probably got other problems that you need to address, like networking issues or other performance problems. And sometimes you may need to specify the solar version itself as an override, but usually the determined automatically setting works just fine. And then at the very end, defaults here work as well. We're not going to get into them, but encourage you, if you're curious, to dig into that a little more. Okay, so if you got your server set up properly, this is what the, if you click the view tab, this is what a successful connection will look like. You don't see any error messages. You may see a message that you need to upload the correct solar schema files to your server. If that's the case, those files are found within the search API solar back end module itself. It's in a, it's in the solar dash conf directory. There's like three different versions. Make sure you choose the correct version for the version of solar that you're running. And if you don't know where, how to get those files up to your host, check your documentation or use their support to find out where those files need to go. Okay, next we'll go up setting up, setting up an index for the search. So the index is what, where all the magic happens and connects, it runs within our solar server. So this is where we'll select the types of content we want to search, choose the fields that we're going to search, and configure how we process inputs and outputs. So back on the main search API configuration page, click add index. And then on the resulting page, give your index a name. So yeah, just a name field there. Then a little bit further down the page, we have our data sources. For our example, we're just going to index content, aka nodes. And then a little further down, we have some more logic for including and excluding specific bundles. We're just going to use all of the bundles that are available on our demo site. And yeah, you can also configure multi-language settings here also. And then further down still, you can choose index order. We're doing first in, first out. You can select the, make sure you select the search server we created in the previous step. And make sure that the index is enabled. And then on our demo site, we have it set to index items immediately. This could have an impact on performance, so you may not want to do this on production. However, do keep in mind that if you do choose to run your indexing during Cron, there may be a delay for freshly updated content before that's reflected in the index and the results your users see. Okay, next we'll talk about the fields. This is where we add fields from the content types to get indexed. There's a fields tab in the index settings, and we can add individual fields here. Note that each field needs to have a type selected. They're over there in the type column. Any fields you want to be indexed or searchable should be set to use the full text type. You can also choose string for plain text fields, but full text would work fine for those as well. On indexable fields, you can also choose a boost value. This is used to increase relevance for particular fields, so you can use this to tune the search results and skew them towards what's important within your content. Other fields can be added. That don't get in, that aren't part of the actual search results that get returned, but you can use them as variables within the search view. And this is where you'll also add fields that you need to use as facets. And for these, the type should generally match the type of data that's stored in the database. And yeah, there's more information on data types. There's a field set in the bottom. There's many dated types. I haven't used them all. I don't actually know what they all do, but there's lots of options there and I encourage you to experiment. Okay, add a new field. So we're back here at the top. There's an add fields button. Just click that. It'll bring up a modal dialogue. Here you'll select the field you want to add. So like in this case, body. Just click that and click done. Also on this more complex fields, you'll see like some of these have a little plus mark next to them. Those are more complex fields. You may need to actually drill down in certain cases to get to the data that you actually want to index. And speaking of that, this is kind of where we get the paragraphs. Paragraphs are complex field types. For those unfamiliar with paragraphs, it's a module that allows you to create structured content and compound fields. Adding paragraphs to fields, paragraph fields to search API. It can be kind of a pain using that interface we were just looking at because you do, there's you kind of go down this like endless rabbit hole of dependencies. And basically your configuration page can start looking something like this. But there's a little trick and we'll look at an easier way to do that. So what we do is go to your manage display tab for the content type that contains your paragraphs field. There's already a search index view mode available through installing search API. Enable that and include your paragraphs field there. And you can control how it gets rendered. You want it to render as rendered entity, which I believe is the only option. And you can also add other fields that you want to index along with the paragraphs field in this case. And then go back to your search API index configuration for the fields and add a field again like we did before. Only this time you want to choose rendered HTML output. And then that will the next modal that pops up. You can choose the user role that you want to index the content as in most cases. This would probably be anonymous users because those are the people that are going to be viewing your search index. And then also for each of your content types that you're indexing, you can choose a view mode. And this is where you want to choose search index. That's the one we just set up on the article content type. So this is just kind of a workaround. So this fully rendered content will get indexed. There are some drawbacks to this if you're doing... So you can conceivably do all of your indexing this way. Add all of your fields this way. You do lose some ability to boost individual fields because it's all kind of mung together within that. But that's another thing you could consider. But also another way around that you could again add those fields individually in addition to the rendered content within the index. So once our fields are set up, we'll want to manage how they're processed for the index. We do this with search API processors. So you go the last tab there is the processors page. There's lots of options. And we'll talk about some of these in a second. But first let's talk about the different kinds of processing. There's pre-process index. This stage occurs before content is sent off to solar to be indexed and stored. The next stage would be pre-process query. This happens when search request is sent to the server. Processors in this stage, they can manipulate keywords that are sent to solar. Generally the ones used during pre-process query are the same ones used during pre-process index. This kind of ensures that the input equals the output ensures better matching. And lastly, post-process query. In this phase we can alter the response returned by solar. So typically you'll only see highlight here because that's kind of post-processing that happens that highlights the keywords that the user is searching for in the search result itself. So now let's look at some of the common filters and the ones we have set up on the demo site. First is highlight. That's the one that adds the keyword highlighting to return to the return search results. It also allows you to define the length of the excerpt that's returned, whether you're even creating an excerpt and allows you to exclude fields from that excerpt as well. HTML filter is another common one. It strips tags from the index content, which is a good idea. It also allows you to boost relevance on specific tags that get stripped out, but that allows you to further tune your search results along with the ones that the boost settings that were set up for the individual fields. Ignore case. This can improve matching too. It does exactly what it sounds like. It basically lower cases all of your search terms that get indexed and also the terms that get sent from your search request. This is stemmer. Stemmer uses an algorithm to reduce words to their root. For example, the words like nerdy or nerdish would get reduced to just nerd. This increases the likelihood of finding a match. Stop words. Stop words removes common words that don't provide meaning to the content. Words like if and but, a, is. This will help performance as well because it reduces the overall size of your index. And lastly, tokenizer. So tokenization breaks up content into individual words. It's a must have for search to work at all. Your search backend, including solar, may also provide tokenizing, but you'll generally get best results by enabling this processor anyway. There's others available. You should experiment with various processors to get a better understanding of how they affect your search. Also pay special attention to the order that the processors run in within the various stages of processing because that can affect how accurate your results are. Okay, so finally we have our index setup. This is the index status page. Here you can see that everything's ready to go. This is also where you can go monitor the status of your index and re-index your content manually if needed. So now we've got our search API server setup. We've got our index setup. Now we need to create the view that our end users are going to use. And so this is all just views. We go create a new view under view settings. Choose your search index that you created as the base for the view. Under the page settings, we'll just create a page. You could also do a block and include it some other way. But yeah, we'll include a page that displays an unordered list of fields. Save and edit. Oh, there's our thing. Save and edit. And then check out the demo site for details on... Wait a second. Okay. Yeah, the demo site will have more details on this. Okay, so for our demo site we also wanted to have an excerpt returned even if a highlighted excerpt wasn't returned. So what I mean by that is there may be cases, probably uncommon, but if let's say you had a keyword that a user was searching for and it only appeared in the title but did not appear in the body content, probably unlikely but could happen, you might get the title returned, but if there's no search excerpt that gets returned with that keyword in it, it would just be blank and that might look a little weird to your users. So we'll set up a couple fields that provide an excerpt that doesn't have the highlighted word, just a typical teaser type limited string of the content. We'll set those up here. Again, look at the... You can download the demo site and see what we're doing here, but we add those fields and we exclude them from the view display and then we'll add them in the search excerpt page in here in a second. So next, add the search excerpt field and in the settings for that field, you want to check use highlighted field data. And then if you look down here under our no results behavior, this is where we're rendering the defaults that we set up in the excluded fields above. So if the search doesn't return an excerpt, it will return our little snippets that we're generating from the content itself. And then finally, we'll add our search form. This is just an exposed filter and within the settings for that, you would want to make sure the filter is exposed and that it's required and set the operator to contains any of these words. And then a little bit further down for parse mode, we've selected multiple words. The last thing we want to do is set our search relevance. So this is a field that gets calculated when the search is indexed. And this is based on all of our boost settings that we set up for the fields and also for the HTML filter. And it will get higher ranked sorting descending, we'll get the higher ranked content at the top for your users. Okay, now we'll talk about adding a facet for our search. So there's a facet configuration page. Go there, click add facet. On the add facet page itself, you'll select the source. So this is the view. The source will be the view that we created. Next, we'll add the field that you want to use as a filter. In this case, we're using channel, which is a taxonomy term. If you don't see the field, you want to make sure it's been added to the index and that your index is up to date. Otherwise, it won't show up here. And you would update your index by going back to that index status page and deleting the index and rerunning it. And then lastly, give your facet a name and save. The next screen that comes up will be the settings for it. Select the widget you would like to use. Here you have options for check boxes, drop downs, links. You can do sliders. Sliders are a little finicky. We tried to get one going for the demo, but had a little trouble. So, but that's an option. Reportedly, you can get them working. Let's see. So yeah, select the widget you would like to use. And also you can choose if you want to show the number of results that would be returned by clicking the facet. That just shows up in a little parentheses next to the facet itself. We'll see that when we see the final result. So facets have a ton of settings options. We're going to go again, in the interest of time, gloss over these. We'll cover a few worth noting, but also encourage you to experiment here and see what you can do. Dependent facet is kind of an important one. You can make your facet dependent on the presence of or values in other facets. This is useful for drilling down through nested data. For example, filtering by location where you might want to refine by state and then a city within that state. Transform IDs is typically good for things like taxonomy terms where you want to make sure that the facet label that gets rendered is the name of the taxonomy term instead of the taxonomy term ID. A little bit further down, give your facet a simple alias. This will appear in your URL query string. So make it friendly and short. And then also probably want to hide your facet if it's empty because otherwise it's just there and it doesn't do anything. We'll you can set up your sorting and then you can also and then yeah, you can set up the sort order for your facets and then save. So there's our facet. It's been added. Now we need to place it on our site and we just do this. You can do this through the Drupal block system layout. If you're using something like panels, you could also add it that way. But simply place the facet block in the desired region. And facets are contextually aware. So they will only show on the page where the search view is rendered. So you don't have to worry about getting all fiddly with the visibility settings for the block. And that's it. This is what our search results page will look like. Notice that we're searching on the word kitchens. And then we have snippets that are showing up in our results with the word kitchens highlighted. That's our highlighted excerpt. And we have our facets over there on the left. And then here's a facet that's clicked. We can see our results are reduced to just articles. And that filter by channel is a dependent facet that only shows up if article is selected because channel, in our case, is only relevant to the article content type. And that's it for me. And back to Adam. All right, real quick. Just wanted to touch on monitoring your search. So this is a pretty important piece. So when you first start out and you build your search, you kind of have a general idea of what you want to throw out there for your users to use. You want to keep probably track of how they're using their search. Maybe even gather some information, get some user input in some fashion. So a few ways to kind of do this initially is to take advantage of Google Analytics. It's probably your best option at this point. So it's going to be reliable. You can actually go into Google Analytics itself. They have an option in there to define a query parameter. So you just go in there and set that up based off of your search queries. There's also the Google Analytics module. You can download it and install in Drupal. There's a mechanism in there called track internal search. And that should also assist with keeping track of how your search is being used. There is a community module called Search API Stats. At this current time, it only provides your top search phrases, which can be handy from an admin perspective. If you have an admin screen and you want to see on regular use, what's your top searches? So that's something you maybe want to check out. And there is actually a few other modules that are in early development. So maybe keep an eye on those things. But it's an important thing to note. Just keep track of how your search is being used by your users. That way, you can improve it over time. Using a lot of these options and stuff that we kind of glossed over, as Jeff mentioned, there's a lot inside of Search API's configuration. So lots to take advantage of there. And obviously, you might come around some custom solutions that you might need to have built as well. So here, just to kind of wrap things up, we got at the end of our slides, we got some resources available. One important thing to note is the first resource here that we mentioned is actually an install of the walkthrough that we just went through. And we chose not to actually do the live demo because we wanted to make sure we could see everything go through. So go ahead and go out and check that demo out, run through the install process. It'll actually give you all the content and everything that we have in our live demo. Feel free to play around with it and mess around. If you guys have questions, please ask. You can reach out to us and we'll be happy to help if you have any questions there. As well as we link to the contrib modules that we've used for this presentation. And then we got some documentation there as well. And we also have a few more resources, just from some of the other points that we made throughout the presentation. And then we also have a couple things there for Elastic Search because we feel like that one's an important one to mention because of how powerful it is and it is a great tool. So keep your eye on that one. Big thank you to the maintainers for providing this kind of functionality. This is, there's a lot of work put into this. So I just wanted to give a little shout out there. And just a reminder to go and check out the contributions sprints this Friday. And we'd like to know what you think. So please go out and check out this session on the Drupal.org Nashville website and then take the survey and let us know. So any questions? Feel free to come up to the mic and fire away. We'd be glad to help. And otherwise, if anybody wants to take a look at the demo site that we have set up, we can bring that up at this time. All right. Well, thank you for a wonderful presentation. I'm curious if you can share some thoughts on managing access control, especially custom ones in the search. And the other question would be a bit different, perhaps, is how would you configure something for a regex like searching for media? Oh boy. Maybe that's all. Those are some deep ones. Yeah. I don't have a good quick answer on that one right away, but I mean, I can see where you're coming from because those are some tricky things to deal with. Maybe I think what might work best if you maybe have an explicit example of what your scenario is and you want to shoot it over, then we can connect afterwards and help you maybe get that figured out. Perfect. Probably for more complex things like that too, you could write your own custom processor for the search index. That's probably the way to go about that. So this would be in core or this would be search API plug-in? This would be search API. You can create a custom processor for that that has the logic you need to exclude the results that you want to exclude based on whatever access control parameters you need. All right. Perfect. Thank you. Thanks for the talk. Very good stuff. I ran into a bug. I don't know if you've run into this with the highlighting. And it works fine with words. You know, just you type in words like in your example there. But when you use quotation marks for a phrase, it would return the results correctly but not highlighted. Have you run into this problem? I have not. I wonder if the quote maybe interferes with how it is returned or maybe I got to work out some way of escaping that? Yeah, I'm not sure. We are, I think we're also using display suite. So I'm not sure if that is messing with it. I know you guys used straight views. Excuse you. Maybe the next step to try to see if that goes around the problem. Yeah. And if not. I was curious if you had that. Yeah. If not, just I definitely check out see if there's an issue on the module page itself and create one. If not, if you think it's related to search API. Okay. Thank you. Good start though. Definitely try it on views and see if you get the same thing or not. Great information. My question is setting up a solar search with panels going to be something like using paragraphs. So you would want to search the panel pages themselves? Because it's not in the way I tried it before it wouldn't index the panel pages because they weren't actual notes. Okay. Haven't done that either. You would probably, if you can't get what you need from like rendering from the rendered content on the content types itself, if you have like multiple content types within the panel, you might need to find some way of rendering the panel content and then including that as a rendered entity. I don't know if there's a module that can help with that, but that might also be some custom work you could do. That's what it sounds like. I actually, you might have. I've done that before. Okay. There is still is. Okay. You can use that or to send a document off to the bird in this module. Is that, is that, is it? Yeah. That does, yep. So then you would send, you would send it off as its own like document. So it's not indexing a note, it's indexing, like. That's great. Thank you. Yeah, very cool. Great, great walkthrough. Thank you. I think this gentleman just answered our question as well. Okay. Awesome. Thanks. Well, he didn't answer mine. No, I was just curious. When it comes to dev staging in Prod, would you advocate having three separate hosts, solar hosts and an index on each or put the three indexes on one host or would that vary by scale or any thoughts on that? You definitely want to consider the scale for sure. But yeah, I'd separate them out in some fashion and if you probably keep your production completely separate. So you could, you know, if you have like dev, dev staging environment, test environment, two separate ones there, those could probably be on the same instance and then just keeping separate servers and indexes that way. But I would keep a total separate one for your production stuff. Yeah, that for sure. Also, I think, if I recall, like working on Aquia in the past with this, it seemed like their dev test live environments used the same index if I'm not, yeah. So you want to, there is a setting. No, yeah. Good point. And so that's a setting on the index setup itself. You can set up for each of your environments and the configuration. Whether you want it to be read only, you just need to make sure that that configuration gets enabled for the proper environment and then you could use the same index in that case. I'd love to know a little bit more about how it works with paragraphs specifically. Like in the search results, will that show up as well as the page that it's a part of? So you're asking if you could have the paragraphs a part of the, basically the summary that's in the result? That detail? Well, if you added to this whole search system, like will it show up as its own results in addition to the page at a time? Or I guess I'm confused. No, it should be, it's the page result. So it's basically, it's not a separate thing. You wouldn't, it wouldn't be a separate, like the paragraph itself. So actually there's, when you're setting up the index, there's actually an option to select paragraphs in there. Don't select it, because that would give you that scenario, I believe, is what you're talking about. So what you want to do is you focus on the node itself and the display of that, the rendered output of that node, which should contain all your paragraph content, right? So that's what you're, does that answer your question? Okay. There might be a use for indexing the paragraphs themselves. I don't know what that would be off hand, but. Maybe for an admin tool or something. I could see it that way. Thanks a lot for the presentation. A couple questions. Do you have a recommendation for a tool to set up autocomplete? Yes, actually there is a module for search API that will allow for search, for autocomplete. Okay, cool. I don't know, is it Drupal 8? Ready? Okay, cool. Ready to go. Okay. It is. And then sort of a set up question. Do you, you had mentioned this a little bit. Do you typically do a different index for every content type or when do you choose to make multiple indexes? Yeah, you kind of want to gauge that. So you couldn't have one index. Keep in mind the size of your site, maybe traffic that's coming, all those kind of things, but probably set up different indexes, especially if you have, let's say you have like 10 content types and one content type in particular is like a bulk of your content. Separate that one. You could maybe get away with having the others all contained together if you only have a few nodes per item there. So you want to consider the amount of nodes per content type that you would have in those instances and then look at separating those out. And then over time, you think about, if you notice performance issues or anything like that, then you can set up, separate them out if you needed to. It doesn't hurt sometimes to just separate them anyway. Yeah, that's, if you have the availability. Yeah, that's what I've been doing and then I'm wondering if I'm being insane, maybe six indexes. Yeah, there's probably a certain point where that does become insanity. I, you know, search is, the solar's pretty performant. It can handle a lot. I would maybe start out just, you know, indexing the content types you need for a specific search view itself. So if you just have like your main site search, just I would start out, unless you have a lot of content and a lot of traffic, just index everything under one index. And then if you begin to see performance issues, you might think about splitting it out at that point. All right, all right, thanks a lot. No problem. Hi, I've got a, I think it's a solar configuration question and it has to do with the suggestions that when you do a search, sometimes you get that suggestion in terms of it always seems like the phrase that they return site, do you mean such and such? And it always seems something ridiculous. Doesn't seem to match up with any of our content on our site. So is there a way that you can kind of, can you configure that to point towards your search index? Because right now our solution is just to turn it off because it just seems like kind of, it seems rather comedic. Yeah, so I know like there is a configuration for solar itself that allows that to happen. At the index level, I don't know for Drupal 8, if there's a means of getting that returned in any useful way that you could use in your results. So how are you doing it currently? Is it, are you using that or do you are using a specific module for the? When I've done some research in a while, it seemed like we had to work directly with solar itself versus there wasn't any kind of interface in Drupal for that. But at least in terms of what I tried to assume to help at all. Yeah, and that might be a case like, I don't even know if like some of the, like if you can get that information from some of the major hosts, that might be a solution where you need to self host and really be able to like get into the back end of your solar server itself. Seems like this may be issued due to a stemmer. So I'm curious if the solar search engine itself has a choice of selecting a stemmer or choosing a different one or disabling it altogether? Yeah, you can enable, there is configuration within solar for stemming as well. So yeah, again, like having a custom server, you might be able to get in there and tune it more than you can just using the processors that are available through the Drupal search API module. Okay, thank you. Hi, I was wondering about, you know, fuzzy search. Does the search API have anything for, you know, when somebody misspells something, you know, give related results? Yeah, I personally, I haven't messed around with something like that. To me, it might be a custom solution there you're looking at. Because, you know, with the elastic search, I know there's a fuzzy search. There is, yeah. I was wondering about solar. Yeah, I don't know, actually. To be honest, I don't know. I don't know either. All right, thank you. I was wondering, with the view that returns results, can you mix two indexes at the same time? Is there a way to do that? Or do you have to search each index? I think you can only do one per, yeah. Okay, thanks. On our site, search is probably the most used and often the most hated function on our site. And there's, my question is that there are any developer tools when somebody says, I searched for this and I got this or I searched for this and I didn't get this, and try and understand the relevant score and is there anything that we can go in like, why is this showing up or why is this not showing up? Yeah, I don't, that's probably, I don't know, something probably more on the solar side. Wouldn't it be like from the? Yeah, I don't know what kind of logging it keeps. If that's a problem, you might, if it's not a big deal, it probably is, but look at Elasticsearch because there is a lot of tools for like, looking at monitoring and statistics and you do like data visualizations and things like that. You might be able to get more insight into how users are using your search and the results are getting back. I don't know that solar has that level of. Yeah, it's often a mystery as to why things are showing up, why they're not showing up. Are they not being indexed for some configuration reason or is it that they put a frump instead of Trump? Yeah, yeah. Is it a user error or is it a database error? That's what I'm trying to get to. Yeah, there's just no tools to look at the solar index to say what's in there. I wonder if there's maybe a way you could use Google Analytics to see at what point users are leaving your search page or leaving your site when they're getting frustrated and see what terms they used. I don't know, but that might be an option. At least help you point in the right direction maybe. Great talk. I'm curious if you've ever run to a problem where the excerpts that come through are all lowercase and stemmed and it looks like the processors might have like ran on the excerpts. I don't know if it's because my processors are like out of order or... Yeah, I would check that first. Yeah. Okay. And you think the order could definitely dictate that? Because I was just curious, why would they run on the excerpts in the first place? That sounds weird. Yeah, I've never ran into that kind of an issue, I guess, to be able to pin it down what that would be exactly. It seems like somehow your stemmed words are getting... Wow, yeah, I don't know actually. Sorry. Okay, I think a couple questions ago might have answered this, but so our situation is we're indexing an external source, an external website, it's not our domain, and so it's in a separate index, and we've been trying to figure out if we could possibly combine that in some way with the index from our domain. Okay. We're using NUTCH for the external index. There's probably a way to accomplish that. I don't... Without like getting in the middle of it and seeing a way to get those things to be combined, but similar to when you... At least in this way we set it up, we target a single index to display the results of that index. And I think isn't there... I'm pretty sure there's a way to, on solar side, to combine those indexes together and then return it into one index. But I don't know for sure from an outside source if that's a big ability or not. Yeah, I can't create a view for this that would combine them, right? I don't think so, no. I know. Yeah, because views wants to target a specific index. Right, okay. Multiple sources are hard. I think something we've done in the past in a situation like that is actually create a solar index and use a web service on the external content to get that content in and index it and index it along with a link that goes out to that external content. So basically you have an additional solar index that indexes that site content. Does that make sense? Sort of, I can deal with the views, but I should deal with the solar... Yeah, so with views itself, there's not a way to do more than one. Hey, so we're running a Solar 5 something and every now and then we get duplicates. And I don't know why. Have you ever come across that and what did you do to troubleshoot it? Do you look at results in the indexes? If I... So we have a self-hosted solar so I can go in there and run a query in the URL and get a JSON back. And I'll see that it has the same nids but different hashes that the internal for solar. But it's the same one for that particular knit. I don't know why there's multiple ones. Sometimes there's three, but most of the time it's just two. And then we just clear the... We just delete it and re-index it and then it goes away until it comes back again. And then it rebuilds itself up. Yeah, and it's... There's no pattern. I don't really know if... It's not like a specific content of no type or anything like that. Have you maybe tried to manually redo your index and see if that's when it happens? So you have your index maybe when you clear it out and you set it, then just try to redo it again and see if all those duplicates show up. What that would maybe mean is that it's not replacing... They don't show up when I... So it must be happening on CRON because whenever it's running on its own, if I re-index it myself one time, it's good. So maybe it's happening on a resave. Somebody's editing a node and it saves and then maybe... It's keeping the revision or something maybe like that? I don't know. Yeah, it's possible. It's definitely not a revision. It's identical. Like I can move through all of it and do a diff and there's no difference at all. There's no changes. All the timestamps are the same except for the ones for solar itself, like the last index and whatever. Yeah, it's really weird. That is an odd problem. I don't have a good answer for you. Sorry. Okay. Thank you. Yeah. I'm wondering about minimum word length indexing and where that setting is and does it work? Yeah, it works and let's see. I think it's in... It sounds familiar. Yeah, it's in here somewhere. It's beyond the server side. Yeah, you can set it. Gosh, I'm trying to remember where it is. Okay, while you're looking, a second question. Is there a module that helps solar become a learning search system? So that if the third result of a query is constantly clicked, it moves up. Because I don't understand why solar is so popular. It doesn't learn. There's no fuzzy logic. And really the biggest way we've been able to boost results is using an XML file, the Elevate XML file. On its own, it seems like kind of a basic unimpressive search. Yeah. I don't know, personally, of anything explicitly to address that. But I'd imagine there's got to be some way to do so, probably on the solar side. I don't think you're going to get that on the Drupal side. Okay. And I haven't dealt... Myself, I haven't done a deep diving on the solar side configuration to address things like that. It showed up when you were configuring the index and now it's all clicked in. So you're going to create a new index. Maybe that's what it is. To get the word there? The word limits, yeah. I know I've never seen it. So it doesn't have to do a re-index anyway. Yeah. There is a setting for minimum word length for the life of me. I can't remember where it is now. I've come across it a million times. But it does work in my experience. Maybe it's before. Oh, it might be a processor. Oh, that's maybe where it is. It might be in tokenizer. Here it is. Yeah, it's in the tokenizer. It can change your results. And a lot of the filters will have, especially if you look at the help here, like stemmer, it'll tell you that you wanted to make sure it runs after tokenizer in this case because you don't want to... Yeah, I think with the basic ones, if you just... It should work as advertised. If you follow those directions. But yeah, it certainly play around with the order because it does affect how things get indexed and it does affect the results that get returned. Well, I'm going to say I'm still on Drifl7. But my question is maybe you can answer it. We're currently using the Apache Solar module with Solar on Aquia. And I'm wondering if we need to create a new... The index that we have now that would be compatible if we recreated it or the setup and search API. I would definitely recommend doing that to get a lot more performance and capabilities out of it. I recently actually had to do that for a client. It was replace everything with Apache Solar on Drifl7 site and replaced it with search API and Solar modules. Got much better performance and a lot more options to deal with annoying issues that were there before. What version of Drifl are you running? Seven? Okay, yeah. Yeah, it's definitely an option. If you're not having any problems with your search, maybe take an if it ain't broke, don't fix it kind of approach. It's a little broken. If it's a little broken, then yeah, I would recommend... I would consider replacing it. Just, it can't hurt, right? And you might find that you're getting better results and that sort of thing too, yeah. And it's very similar setup with what we just did here with Drifl7, so it's not too painful to get it moved over. Thank you. Thanks for the presentation. So we already configured the search to show the index results for the nodes. And we also have a few taxon term pages that has to be indexed. And can we show both of the results in the same search view? You should be able to. Yeah, so when you set up your, let's see... Oh, so we are getting the results for the nodes. I mean, indexed results for the nodes. So we also have the taxon term pages. So can we show both of them in the same... So when you set up your index itself, when you choose your data sources, you can choose content and taxonomy terms both. Okay. And then you'll have access and then just... But the taxon term doesn't have the XF, right? The search XF. The search XF won't do for taxon terms, right? I mean, I kind of... You could add that as a field. You could add that as a field. And you could add a... Well, you could add a... Let's see. It seems like this... It doesn't have an option for search excerpt? I mean, it does have... It does for the nodes, like not for taxon terms. So there's... Actually, there's a couple of different ways. Like if you look here, like we're not even using... In the server setting, let's see... I think it's here. That's actually a good question. I don't know. It seems like that should be... Can we do that in the view? Like, so can we show that in the same for the nodes and the taxon terms? Let's see. It's using that same... We'll be also using the taxon term based... I mean, the data change is using the TID, you know? Let's see. Let's go to our index. Oh, we're not doing it. We're not indexing taxonomy term, though. Gosh, it seems like you should be able to... I just see the title. That's it. Like, and also, like if there is any node... If there is a node related... Like, if there are category related nodes, then we can only show the taxonomy, which is the specific node, you know? Like, below the title, that kind of thing. I couldn't show that separately. Is it kind of like multi-site indexes? You might be... So if you're not getting that result, you might want to look at some way of doing some... You must have some kind of... It's a full page display, right? For your taxonomy terms, is that right? So you got to have some kind of content there, where what you might want to do is look at pre-processing the output of that, grabbing the content that you want to be, and then combining it with that title field, and then targeting maybe the search or something like that within the pre-processor to display that together. That might be one way to handle it. So it'd be some custom work to get there. We're on D7, but we used ViewModes. Yeah, I'm using the ViewModes to spit out our search results. And we have one content type, where the title is not set to the actual title. It's like set to a code for this particular... It's a course. So it's set to the course code, and then there's a separate field that's set to the course title. But in the ViewMode, as the results come back, we look to say, is this a course? And if it's a course, we display this other field rather than the title. So if it's a taxonomy term, you could do it in the ViewMode, and say, put something else in there, other than... Instead of doing it in one view, you could say, instead of showing the extra, like you do on a node, you show a description field for a taxonomy term. So you can do some PHP work in the ViewMode itself. Thank you. Yep. Thank you for the presentation. I was only able to capture or see last half, sorry if you covered this already, but boost by age. If you guys could talk a little bit about that, and how would you manage the penalty that you can apply to documents that are older, and maybe setting something along the lines of a minimum, penalty that you can apply for documents, and have that level off after a certain number of years, months, et cetera. I don't know if there's a configuration that's built in to deal with that explicitly. That sounds like you might have to come with a custom solution. Custom processor. You could probably write a custom processor that you could... Look at the published date or something like that, and then have it not available to the index, or decrease the boost on it because of that. I don't think there's a way to do it through configuration. That sounds like custom work to me. Okay. Second question. Omit norms. Is that something that you could talk about too in terms of configuration, and how you can apply that to some of your content? I'm not familiar with that. So my understanding is that you can essentially...