With the release of Couchbase 6.6 the Eventing Service has major improvements in functionality.

We’ve introduced new Eventing Timers that can be cancelled using the cancelTimer() function, or by creating a new timer with the same reference identifier as an existing timer. Recurring timers are also fully supported, and can easily be used to create repetitive logic by using a timer callback to create fresh timers. Timer scheduling enables timers to be created for days, weeks, or years in the future with no adverse performance impact. The OnDelete Handler now indicates if a document was deleted or expired using the new “options” parameter. Key Eventing statistics in the UI are now co-located with each Functions lifecycle control.

Together these enhancements simplify the effort and code required to make robust business logic.

Prerequisites

In this article we will be introducing key Eventing improvements added to the latest GA release, i.e. Couchbase version 6.6.0, and for each item we provide a working basic example.  However, please understand that none of the Eventing Functions provided in this article will work “as-is” in earlier versions of the Couchbase server without significant changes and complex work arounds.

If you are not familiar with Couchbase or the Eventing service please walk through GET STARTED and one Eventing example specifically refer to the following:

  • Setup a working Couchbase 6.6.0 server as per the directions in Start Here!
  • Understand both basic Eventing concepts and how to deploy a basic Eventing Function as per the directions in the Document Archival example.

Eventing Timers can now be Cancelled

With the addition of the cancelTimer() function, or by creating a new Eventing Timer with the same reference identifier as an existing timer active Timers that have yet to fire can be canceled.  This enhancement simplifies the code required to make robust business logic.

Developers no longer are forced to add fields and logic and do additional checking to ensure a Timer being fired is not “stale” and superseded by a newer Timer.

Example:

  • Create a “source” bucket and a “metadata” bucket for Eventing.
  • Deploy this function (code below).
  • Create a document in the source bucket with:
  • Inspect the logs after a minute.
  • Delete the document with key “user_scoreboard::1”.
  • Inspect the logs after a minute.
  • Undeploy this function.

Recurring Eventing Timers are now fully supported

Recurring Timers are fully supported, i.e. a function that is invoked by a timer callback can reliably create fresh Timers.  This update allows a single Eventing Function to reliably implement complex recurring events creating new timers from within the callback of another timer.

Previously a co-Function was required to reliably create a recurring series of Timers (left image), the 6.6 release simplifies the code needed to implement recurring (or scheduled) business logic (right image).

Eventing cron update

Example:

  • Create a “source” bucket and a “metadata” bucket for Eventing.
  • Make an Bucket alias in the Function’s setting as “src_bkt” to the source bucket in mode read+write.
  • Deploy this function (code below).
  • Create a document in the source bucket with:
  • Inspect the logs after a minute.
  • Inspect the logs after several minutes.
  • Alter the document with key “recurring_timer::1” and change the field “active” to false as follows:
  • Inspect the logs after a minute.
  • Undeploy this function.

Timers can be created days/weeks/years in the future

One timer or a million timers can be created with no adverse performance impact on an otherwise idle Eventing system. This ability opens up use cases for long term notifications and customer re-engagement programs.

In the 6.5.X releases creating a few thousand Timers in the future (as in one hour+) in an otherwise idle system resulted in a growing number of metadata bucket operations which impacted performance and could eventually block mutations for a the given Eventing Function.[1]

Note on a cancelTimer() operation or overwriting an existing Timer by reference there will be a temporary documents in the Eventing “metadata” bucket, eventually these documents will be purged.[2]

Example:

Demonstrate the creation of a lot of Eventing Timers (in this case 50,000) and schedule them well into the future (by 96 hours) and then cancel the entire set (or allowed them all to fire the callback routine TimerCallback).

  • Create a “source” bucket and a “metadata” bucket for Eventing.
  • Deploy this function (code below).
  • Create a document in the source bucket with any DATA as we only look at.
  • Inspect the logs after a minute or two.
  • Delete the document with key “spawn_50k_timers::1”.
  • Inspect the logs after a minute or two.
  • Undeploy this function.

To see the 50K Eventing Timers being created and firing (instead of being canceled):

  • Edit the function change delayMinutes to 1 (you do not want to wait 4 days).
  • Deploy the modified function (code below).
  • Create a document in the source bucket with any DATA as we only look at the KEY.
  • Inspect the logs after two minutes.
  • Delete the document with key “spawn_50k_timers::1”.
  • Inspect the logs after a minute or two.
  • Undeploy this function.

Eventing Statistics are now co-located with Lifecycle Controls

Four (4) key Eventing statistics in the UI are now co-located with each Functions Lifecycle control. These enhancements simplify the effort and code and diagnose Eventing Functions to quickly make robust business logic.

The developer or administrator can immediately determine that a Function is misbehaving without having to navigate away from the Function’s lifecycle controls.

Example:

  • Create a “source” bucket and a “metadata” bucket for Eventing.
  • Make a few document any KEY and any DATA in the source bucket.
  • Deploy the modified function (code below) it contains a syntax error,  forgetting to put var in front of a variable.

  • When the Eventing Function is deployed there is an immediate feedback that something went wrong to a developer.

  • The user responds to the feed back, i.e. “red” failures and can inspect the Function for the source of the error.

  • Undeploy this Eventing Function.
  • Fix the error in this case make line 4: “var a=2;”.
  • Deploy this Eventing Function again.
  • Once the code is fixed the developer can see the correct behavior and progress as success counter now increments.

OnDelete Handler now indicates Deletion or Expiration

The OnDelete Handler now indicates if a document was deleted or expired via a new parameter “options”.  This often requested capability allows different logic to be executed depending on the type of removal.

Example:

  • Create a “source” bucket and a “metadata” bucket for Eventing
  • Deploy this function (code below)
  • Create a document in the source bucket with any DATA as we only look at the KEY
  • Inspect the logs after a few seconds
  • Delete the document with key “doc_to_delete::1”
  • Inspect the logs after a minute
  • Undeploy this function

Explore Couchbase Server 6.6 resources

BlogsDocs and TutorialsWebpages and Webinars
What’s New in Couchbase Server 6.6What’s New in Couchbase Server 6.6?New Features in Couchbase Server 6.6: Analytics, Backup, Query, and More
Eventing Improvements (Timers, Handlers, and Statistics)Couchbase Server 6.6 Release NotesCouchbase Analytics Service
Remote Links – Analyze Your Enterprise With Couchbase AnalyticsTry the Couchbase Index Advisor ServiceWhat’s New in Couchbase Server (Product Page)
External Datasets – Extend Your Reach With Couchbase AnalyticsSet Up Analytics Remote and S3 Links Using REST APICompare Editions
Announcing Flex Index With CouchbaseCreate External Datasets Using Data Definition Language (DDL)
Introducing Backing Up to Object Store (S3)Set Up Analytics Remote and S3 Links Using CLI
Import Documents With the Web Admin Console

References

We would love to hear from you on how you liked the 6.6 features and how it’ll benefit your business going forward. Please share your feedback via the comments or in the Couchbase forum

Footnotes

[1] The severity is governed by: a) The number of vBuckets holding an active timer. Therefore if there are only a few timers in the future the issue may not be noticeable or materialize. and b) Whether an Eventing timer has fired recently on a vBucket (which clears the issue for the given vBucket on a per function basis). Therefore 6.5 systems with lots of near term timer activity will not experience this issue even if timers are scheduled far into the future.  This is not an issue in 6.6.

[2] On a cancelTimer() operation or overwriting an existing Timer by reference there will be a temporary documents in the Eventing “metadata” bucket which will be cleaned up at a) at the initial firing schedule of the canceled (or overwritten) Timers, or b) an undeployment of the function. This behavior is unlike a Timer firing at the scheduled time in which all associated Eventing metadata is immediately cleaned up form the  Eventing “metadata” bucket upon a Timer firing.

 

Posted by Jon Strabala, Principal Product Manager, Couchbase

Jon Strabala is a Principal Product Manager, responsible for the Couchbase Eventing Service. Before joining Couchbase, he spent more than 20 years building software products across various domains, starting with EDA in aerospace then transitioning to building enterprise software focused on what today is coined “IoT” and “at-scale data.” Jon worked for several small software consultancies until eventually starting and managing his own firm. He has extensive experience in NoSQL/NewSQL, both in contributing and commercializing new technologies such as compressed bitmaps and column stores. Jon holds a bachelor’s degree in electrical engineering and a master's in computer engineering, both from the University of Southern California, and an MBA from the University of California at Irvine.

Leave a reply