Like most B2B and B2C companies, gaining a deeper understanding of customer behaviors and desires has become essential to commercial success. However, customer data is trapped in different systems and data stores, limiting the actionable insights that the business can extract to deliver a better customer experience.
This need was the starting point for a major software development company to implement a scalable, secure, robust, yet flexible Customer 360 solution available to different company roles that impact customer engagement.
Customer 360: data belongs together
Over the past several years, business growth resulted in customer data spread across nearly ten different internal and external-facing data repositories and systems, each owned by other parts of the organization. For example, Marketo handles customer and prospect email communication, Salesforce manages orders and purchases, Zendesk for the support team and numerous spreadsheets and dashboards. It did not take long for these sources to get out of sync and need manual maintenance.
The challenge with the existing data infrastructure was threefold:
- Customers and users want to interact with a single database, and they expect a clear and consistent engagement back.
- The customer-facing teams have to learn how to navigate each system, find the data they need and manually connect the analytical dots between customers named inconsistently across them.
- Cross-customer analytics and BI were impossible.
Continued corporate growth and effectiveness depended on providing more consistent, secure and easy to navigate access to customer-related data. A new initiative was born from this imperative: Build a consolidated, comprehensive Customer 360 repository and dashboard accessible to everyone. The primary mandate was to improve the sales organization’s productivity while also providing insights into product efficiency and future direction.
Real-time customer insights
Queries against this new Customer 360 repository had to be both deep and broad. Some groups need a comprehensive view of all customer-related data for any given customer in real-time. Other groups needed access to BI analytics across all customer bases, their status and direction. To truly meet the mandate’s objectives, the new repository had to satisfy the requirements of both groups.
Modernization vs. custom ETL pipelines
The first approach to these challenges was building an ETL framework using various tools that could move data from multiple sources. The tools moved the data repositories into a flexible, scalable target repository that served as the basis for the new Customer 360 applications and BI dashboards. The target repository was the easy part since they already used Couchbase. The ETL pipeline, while adequate, quickly demonstrated its drawbacks. First, it was not flexible; new repositories or data sources (often coming from Sales and Marketing) needed new ETL pipelines and specialized IT time and resources to build. Likewise, the ETL processes required IT resources to investigate and resolve problems as they occurred. Naturally, the ETL solution didn’t solve the core data quality, governance, security, and reliability issues.
At the end of the first phase of the project, the conclusion was that Couchbase as a database (first Couchbase Server and later Couchase Capella) provided all the necessary scalability, flexibility, and reliability that the customer needed for the consolidated data repository. Unfortunately, the hand-crafted ETL pipeline was not up to the challenge. The “build-it-yourself” approach worked well for existing users, datasets, and queries. Still, it was not good at adapting to new data sources, new users, and new business questions or analytics.
No-code data pipelines accelerate Time to Value
At this point, Dataworkz was introduced to simplify the data management stack. Dataworkz is a SaaS service that unifies multiple layers of the data management stack – data discovery, cataloging, transformation, lineage and monitoring. After a detailed security review, access was provided to the CRM (Salesforce), prospecting (outreach.io), and MAP (Marketo) repositories to begin retrieving data in a matter of minutes. In addition to the speed and simplicity, this approach had several advantages:
- Dataworkz provides data governance, processing, quality and lineage to build operational analytics dashboards.
- Data does not reside in Dataworkz—it is always either in the source or target repositories, providing better security and reducing the overall storage and cost.
- Business users in the marketing team can leverage Dataworkz’s easy-to-use, flexible interface to change data pipelines as requirements evolve without requiring scarce IT resources.
- Dataworkz automatically detects schema changes in source data and sends alerts to relevant stakeholders. These functions streamline collaboration between the operations teams managing the SaaS data (like SFDC), the analytics teams building the Customer 360 application and the marketing teams building executive dashboards.
- Dataworkz’s built-in data monitoring and proactive anomaly detection enabled the creation of resilient pipelines, which simplified downstream data consumption.
An accelerated project cycle
A fully functional Customer 360 application and data pipeline project would take 9+ months to gather requirements, build and iterate over the data flow, add analytics and deploy to the business user. The combination of Dataworkz and Couchbase Capella shortened that process to just three months.
With Dataworkz’s no-code, easy-to-define visual interface, they built the primary data flow in hours rather than days or weeks. Gathering additional requirements and iterating through data flow changes took a few days instead of weeks or months with the traditional DIY ETL approach. Finally, Couchbase’s built-in, native analytics capability shortened the time needed to build out added BI analytics.
The future of Analytics with Couchbase Capella
Applying Dataworkz’s comprehensive data management capabilities, Couchbase Capella as the Customer 360 repository and Power BI for business analytics allowed the organization to enable operational and analytical workloads off a centralized, consistent dataset.
While the initial phase of this project focused on creating a holistic view of the customer base, the next step introduces ML and AI into the business process. Using Dataworkz’s built-in ML and AI over daily snapshots from different source applications during various customer journey stages (prospecting, marketing, sales, and support) allows for transitioning to AI-driven business decisions.
Start understanding your customer better today:
Dataworkz and Couchbase partnership
This post was a collaboration between Couchbase and Dataworkz.
- Sachin Smotra, Co-founder and CEO at Dataworkz
- Perry Krug, Director, Shared Services at Couchbase.