Full Text Search (or FTS) is a new feature that is being released as a developer preview in Couchbase 4.5. It is not meant for production, and is not yet supported, but it is a glimpse at things to come.

When offering the ability to search, a simple LIKE '%foo%' with N1QL is not enough. Your users expect a richer search experience: one that is language aware and ranks results by their relevance.

In this blog post, I'm going to show you:

  • How to set up a Full Text Search index
  • How to execute a Full Text Search query in the Couchbase Console
  • How to execute a Full Text Search query with highlighting in ASP.NET

Set up a Full Text Search index

First, you need to make sure that you are using Couchbase 4.5 (beta, at least), that you have access to the Couchbase Console, and that your cluster is running a Full Text service. You can check to see if you have a Full Text service running by clicking “Server Nodes” in Couchbase Console. Services (like Full Text) have to be enabled/disabled when adding a node to the cluster. If you need a refresher, check out my blog post on how to install and setup Couchbase on Windows.

Server Nodes - Services available

I'm going to setup a full text search index on the travel-sample sample data bucket. You can install this by going to Settings -> Sample Buckets, selecting 'travel-sample' and clicking “Create.”

Create a sample bucket

In this sample data, there are documents of type “landmark”. Among other things, these documents contain 'content' fields. Here's a snippet from one of the landmark documents. (ID 'landmark_16079' if you're following along):

With that structure in mind, let's create the index. Go to Indexes->Full Text, and click “New Full Text Index”.

I'm going to name the index travel-sample-idx, and put it on the travel-sample bucket. There are a lot of options here, but I'm just going to cover type mappings. Expand “Type Mappings”, and click “Add Type Mapping”. This allows you to restrict the full text search to documents of a certain type (in this case, “landmark”, which is matched to the “type” field). Also check the “only index specified fields” box, otherwise all the fields in the documents will be indexed for search. Hit “ok”.

Create full text search for landmark documents

Next, hover over the 'landmark' type mapping, click the + symbol, and click 'insert child field'. If you want to index search data that's further down in the hierarchy of your document, then you would need to 'insert child mapping' (as many levels as it takes) until you get down to it. In our case, the content field is at the root level, so I don't need to do that.

For the child field, enter the field name (“content”) in field, leave everything else as-is, and check the “store” checkbox (more on that later), and hit “ok.”

Create child field for full text search

Finally, edit the “default” type mapping and uncheck the “enabled” option. This mapping would enable searching across all documents, but in this example we want to limit to just landmark. Hit “ok”, and “Create Index”.

Disable default type mapping on full text search

After you hit “Create Index”, you'll be shown an overview of the index you just created. It will immediately start indexing documents. If you click the Refresh button, you'll be shown the count of the documents that have been indexed so far.

Full Text Search indexing in progress

It will be 100% when it's complete.

Full Text Search indexing complete

Execute a Full Text Search query in the Couchbase Console

Before we go into C#/.NET, you can try the full text search right from the console. Go to Indexes->Full Text, and select the full text index that you just created. Enter a search term (if you're using the travel sample, “inn”, “skyscraper”, or “bed and breakfast” are good terms to try.

Using the full text search

When you click search, you'll be given a page of search results (document ID and a fragment of content) in order of their relevance score. Notice that the search terms are highlighted in the content fragment. Enabling “store” on the child field makes this work. If you don't enable store, you'll still get the document results in relevant order, but you won't get the highlighted content fragment.

Full Text Search results in Couchbase Console 

Note that the full text search uses Bleve, and allows you to do more complex searches (exact phrases, boolean queries, etc). Try searching "bed and breakfast" or bed -breakfast, for instance, and see what happens.

Full Text Search query with highlighting in ASP.NET

Once you are satisfied with a full text search index that you've created, it's time to take it out of the Couchbase Console and into a real application. I've created an example in ASP.NET that demonstrates how to use the Couchbase .NET SDK to perform a full text search. For this example, I'm assuming that you're comfortable with ASP.NET MVC and with the basics of integrating ASP.NET with Couchbase. If that's not the case, I recommend that you review the ASP.NET “Hello World” example that I blogged about previously.

I've created two actions: Index and Search. The Index action shows a view that allows a user to type in a search string. Here's the controller and action:

And here's the view in Index.cshtml:

It's not pretty, but it's functional. Next, I created a Search action. This action will take in the search query and output the results from Couchbase. The results will look very similar to what we saw in Couchbase Console: document ID, relevance score, and highlighted fragments.

There are three steps to executing a full text search:

  1. Instantiate some object that implements ISearchQuery. There are many options; I'm using MatchQuery for this example.
  2. Instantiate a SearchParams object to specify search options.
  3. Use an IBucket to execute the query+parameters and get results.

In this example, I've done all three steps right in the Search action:

Some notes on the SearchParams:

  • I'm limiting it to the first 10. There is also a Skip() method that you can use for paging.
  • I'm setting a timeout of 10000ms. If I create the index correctly, the results will be returned very quickly, but if I tinker with the index and something goes wrong, I want the search to fail quickly.
  • I'm setting the highlight style to HTML. This will put the tag around the highlighted search terms in the fragment results.

I've also created a SearchResultsModel view model. This does the mapping that I need to get the document ID, score, fragments from the search results (as well as the keywords that the user typed in).

Finally, here's the Search.cshtml view:

Not a terribly pretty HTML page, but it's functional. Here's what it looks like in action:

Full Text Search results in ASP.NET

Conclusion

Remember that this feature is just a developer preview: it's not supported yet and it's not recommended for production yet. But full text search is coming to Couchbase. In fact, you can have a say in how the feature is implemented. Check out the Full Text Search RFC on GitHub, and feel free to ask questions or leave feedback.

Still have questions? Need help with anything? Leave a comment, ping me on Twitter, or email me (matthew.groves AT couchbase DOT com).

Posted by Matthew Groves, Developer Advocate

Matthew is a Developer Advocate for Couchbase, and lives in the Central Ohio area. He has experience as a web developer as a consultant, in-house developer, and product developer. He has been a regular speaker at conferences and user groups all over the United States, and he has written AOP in .NET for Manning Books. He has experience in C# and .NET, but also with other web-related tools and technologies like JavaScript and PHP. You can find him on Twitter at @mgroves.

Leave a reply