From N1QL to Javascript and Back – Part 1: Introduction

Among the many great features introduced in Couchbase Server 7.0, one has the potential for a greater impact in later releases: N1QL User Defined Functions.

A quick glance at the past

N1QL is a declarative language, meaning that requests merely indicate what the user requires, without ever needing to specify how the request should be carried out.

This has worked very well for query languages, albeit with the limitation of the lack of bespoke business logic.

User Defined Functions bridge that gap.

The complete view of UDFs in Couchbase Server 7.0 is available in this blog, but just to avoid jumping back and forth, a quick reminder of a few important things:

• • UDFs come in several languages. In this blog we are particularly interested in JavaScript.
  • UDFs come in two flavours, cluster wide (or global UDFs), useful when you haven’t yet embraced collections, and scope bound (or scope UDFs), which allow you to have several copies of the same UDF (one per scope), but each with potentially different logic.
  • N1QL does not offer UDF overloading (having multiple UDFs with same name and different signature, each supporting logic specific to its input), but the same functionality can be achieved with variadic UDFs.
  • N1QL UDFs can generate dynamic data which can be used as a source to N1QL queries.

JavaScript UDFs

• • JavaScript is an external language, meaning that the management and execution of JavaScript libraries is delegated to an entity different from the N1QL service. This entity is called jsevaluator.
  • Creating JavaScript functions is a two step process: first you create the actual javascript code using the jsevaluator interface, and then you create the N1QL UDF referencing it.
  • One of the side benefits of the code being maintained externally is that several UDFs can share the same library and thus the same code, and they all get updated at the same time when the code changes: shared libraries can go in step with your application, as opposed to the scope schema.
  • In the 7.0 release the jsevaluator only supports plain JavaScript.

The missing bits

The jsevaluator does not support global states – since, as we have seen, the same library can be used by multiple requests concurrently across the query service: saving and retrieving values in global variables would lead to unexpected results.

The side effect of this is that global arrow functions are not supported, as the function reference is saved in a global variable.

Local arrow functions (those declared inside global function bodies) work as expected.

An actual example

First create the JavaScript library and function:

Then create the N1QL UDF:

Note that it is perfectly legitimate to create the UDF first and the library later, but, until you create the library, you won’t be able to execute the UDF!

New in 7.1!

User interface

You no longer need to concern yourself with strange REST endpoint and curl commands to manipulate javascript libraries: the Query page has a new tab, UDF:

Upon clicking it lists are shown of JavaScript libraries and N1QL UDF definitions:

There are buttons to add and edit both JavaScript libraries and N1QL User Defined Functions.

The important thing to note is that Libraries are being edited or added, not individual functions: to add a new function to a JavaScript library, the whole library must be edited, and re-submitted with all the existing functions and the new function (submitting the new function alone will obliterate the existing functions, so do use care!).

Note that while there are plans to take the UDF JavaScript libraries UI to the same place where the Eventing UI is (e.g. with a debugger), we are not there yet, and the UI will likely change as a result, both in aspect and location.

N1QL in JavaScript

The JavaScript engine shipped with 7.0.x is only able to execute vanilla JavaScript, and lacks the ability to execute N1QL requests.

In 7.1.x the JavaScript engine can now execute multiple N1QL statements and supports both iterators and transactions.

Hierarchical JavaScript library storage

In 7.0 the JavaScript library store is a flat repository, meaning that all the libraries sit at the same level, and are visible to both global and scope N1QL UDFs.

These libraries are shared.

In 7.1 the storage has been extended to be hierarchical.

Root level libraries are still accessible to both global and scope N1QL UDFs and shared as before, and in addition it is now possible to have libraries defined under a specific bucket and scope, either as separate implementations of root level libraries specific to one particular scope, or as a private library storage for users with access to just one particular scope.

Conclusion

This blog gives you a taster of the JavaScript user defined functions features in Couchbase Server 7.1.

We will explore each of them in detail in subsequent blogs.

• • For now, read more about N1QL UDFs in Couchbase