Storing sensitive information in a database without encryption is not a good thing, but have you ever done in on accident due to bugs in your application code or a misconfiguration in your database? What if there was a way to receive notifications anytime something sensitive was detected so that immediate action could be taken?
We’re going to see how to leverage the Couchbase Eventing service to write JavaScript functions that will detect changes in documents as they happen. In particular, we’re going to see if social security numbers are popping up in documents, when they shouldn’t be.
Going forward, it is important to note that this is just one of many possible use cases when it comes to eventing in Couchbase. The Eventing service is available as of Couchbase Server 5.5 which has just been released.
Please note, the N1QL construct is still in Development. This feature is intended for Development purposes only and should not be used in Production environments.
Creating a RESTful API with Node.js and Express Framework
In this example we won’t actually be sending notifications. That is a whole new can of worms. Instead we’re going to create a sample API that the database will ping every time a function is triggered. This API will print out a message, but in reality you’d have it send out a push notification or email.
Create a new directory somewhere on your computer and execute the following from within that directory:
1 |
npm init -y |
The above command will create a new Node.js project, more specifically a package.json file. Our API won’t need to use Couchbase for this example, but we will need a framework for receiving HTTP requests.
Within the project, execute the following command:
1 |
npm install express body-parser --save |
The above command will install Express as well as a middleware for accepting request bodies in JSON format.
Now create an app.js file within your project and include the following JavaScript code:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
const Express = require("express"); const BodyParser = require("body-parser"); var app = Express(); app.use(BodyParser.json()); app.use(BodyParser.urlencoded({ extended: true })); app.post("/notify", (request, response) => { console.log("POST /notify"); for(key in request.body) { if(request.body.hasOwnProperty(key)) { console.log(JSON.parse(key)); } } response.send("sent"); }); var server = app.listen(3000, () => { console.log("Listening..."); }); |
The emphasis of this tutorial is not creating an API with Node.js so I won’t go too much into it. Essentially we’re setting up a single API endpoint, parsing the data that comes in, and are printing it out to the console. This can be done with any technology, not specific to Node.js and in reality you’d probably be sending out some kind of message when the endpoint is triggered.
Now we can focus on what matters, the functions for Couchbase Server.
Creating a Function to Listen for NoSQL Document Changes
Like I said previously, the Eventing service is available as of Couchbase 5.5, which has just been released.
To be successful with the Eventing service as of now, you’ll need two Buckets.
In my example, I have an example Bucket and an example-eventing Bucket. Both are standard Buckets, but one will be used to store metadata.
Since we’ll be making requests from Couchbase Server to our notification API, we need to configure a whitelist because by default, no requests can leave Couchbase Server for security reasons.
In the Settings area of the Couchbase dashboard, look for the Advanced N1QL Settings section. I’m running both Couchbase and the API locally, so I would just need to add the host and port. Your settings might be slightly different.
Now we can go ahead and create the function that will keep an eye out for sensitive data.
When adding a new function, fill out the information to match your Bucket details. The logic to drive the function will come next.
With the function available, we want to edit the JavaScript. By default, you should see something similar to the following:
1 2 3 4 5 |
function OnUpdate(doc, meta) { log('document', doc); } function OnDelete(meta) { } |
Our goal is to trigger a function when a document is updated, rather than deleted. A number of things will need to happen.
- Check to see if we’ve created or altered a
person
document. - Look at the
ssn
property of the document and see if a regular expression can validate it as a social security number. - Send out a cURL request to an API.
The code explaining the above steps can be seen below:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
function OnUpdate(doc, meta) { log('document', doc); if(doc.type == 'person') { var isSSN = /^(?:\d{3}-\d{2}-\d{4}|\d{2}-\d{7})$/; if(isSSN.test(doc.ssn)) { var http = SELECT CURL( "http://localhost:3000/notify", { "request": "POST", "header": "application/json", "data": $meta } ); http.execQuery(); log('ssn', 'sending notification for raw ssn') } } } |
Notice that we are using a N1QL query to issue the cURL request. It isn’t the only way, but it is an option which can be read about in the N1QL documentation.
In our example, rather than sending the document itself to our API and further exposing the social security number, we send the meta information which includes the document key. Then the recipient can just look for the document based on the key.
Conclusion
You just saw how to use the Couchbase Eventing service to create functions and detect sensitive information such as social security numbers. You can create functions to alter data within the database or to send notifications like demonstrated in this example.
If you’re interested in learning more about the Eventing service, check out the documentation via the Couchbase Developer Portal or read the new service announcement here. Learn about other features available in the Couchbase Server 5.5 release.