Self-service data preparation, which we define as empowering business workers and analysts to prepare data for themselves prior to analysis, is often cited as the next big thing. In fact, Gartner predicted last year that “by 2018 most business users and analysts in organisations will have access to self-service tools to prepare data for analysis“.
The great news about the new self-service data preparation trend is that it goes beyond the office of the CIO. It represents a significant step forward for data-driven organisations as a whole. If done well, it will allow anyone to put data to work in their operational context whether they are in finance, sales, marketing, operations or HR.
This is a hugely exciting prospect for any business because up to now, obtaining accurate data capable of furthering an organisation’s strategic goals has typically been a difficult, time-consuming and intractable challenge. Business analysts spend far too much time making data ready for analysis, finding the data source, discovering its content, cleansing it and standardising it so that it aggregates correctly and connects with other data points. In fact, according to Forrester, business analysts spend roughly 80 percent of their time on repetitive tasks that prepare data for analysis, which leaves just 20 percent of their time to extracting and sharing those insights. According to BlueHill research, this makes 2 hours in their typical workday of a data analyst, and translates into a cost of $22K per year. Those people are supposed to be your data heroes that can drive you to untapped efficiency boosters and reach a new level of customer intimacy or new markets, but they are cleansing data. Not only this is boring and repetitive, but this is highly inefficient. Research show that they are poorly equipped with tools; it is estimated that 78% are using a spreadsheet, and 46% rely on scripting or coding by themselves or through their IT colleagues.d
As Philip Howard, research director at Bloor Research states: “Extracting value from data has historically involved a lot of time and effort – especially when it is disparate and from multiple sources. And far too much time and effort have been spent just getting the data ready to be analysed rather than in the analysis process itself.”
Historically, the task of data preparation has too often fallen solely to overburdened IT teams that can’t keep up with the growing data demands of the business, and data analysts who are all too frequently spending more time wrangling data than they are supplying insights. Fortunately, with the advent of the latest open source self-service applications, this is now all starting to change, helping to bring Gartner’s prediction closer to fruition.
One of the great advances delivered by self-service data preparation is that it extends the benefits of digital transformation to all business users and not just a few analysts and IT experts that previously understood it. The ability to explore, cleanse, enrich and combine data in minutes instead of hours allows line of business users to apply their own unique domain expertise and work directly with the data that’s relevant to their business objectives. By simplifying the whole process, this kind of capability represents another step towards the democratisation of data analysis and a further step away from using traditional spreadsheets. Data analysis was once a highly specialised task, which an only expert could undertake.
These latest advancements help to empower executives from lines of business across the whole organisation – from HR professionals to finance directors, sales teams and marketers – even those with no IT background at all – to get data into the format they need and use the results to advance the operational goals and strategic objectives of their departments. Through modern technologies such as in-memory data discovery or machine learning, users can start interactively working on their data through a guided process while avoiding having to create complicated formulas, write code or complete the same tasks over and over again.
That does not mean that the IT department no longer has a role to play or that specific business lines should work in isolation, however. The best self-service tools should foster collaboration, effectively providing a sound way to reconcile IT and lines of business so that they can unlock data collaboratively. Data democratisation is not anarchy. It needs coaching, collaboration and guidance, together with control, rules and governance, otherwise, it will fail.
Delivering Value to Every Department
The good news is that the best of the self-service data preparation tools emerging today have built-in data governance and with the help and guidance of IT, will allow business lines to get up and running quickly—opening up data lakes to more workers. Stakeholders that are in charge of provisioning, managing the quality and securing data assets, such as Data Architects, IT developers or data stewards can take handle of data governance, data control, and reuse. They can deliver sanctioned data as on-demand data set, exposing the right data to the right people at the right time. And they can as well crowdsource data preparations rules designed by the business user that best know their data, operationalize those rules and publish them across the enterprise.
The ability to use the latest technology to streamline the whole process and reduce the time taken to prepare data from hours to minutes will help departments wrest back time from the laborious process of cleaning and crunching data and allow them to spend it to turn critical tasks into data-driven activity like personalizing customer experiences, streamlining the collection, transformation and publishing of financial information, reducing the cost of compliance, or increasing the conversion rate of incoming sales leads.
In short, the ability of the latest self-service technology to help various line of business analysts access, cleanse and prepare their data quickly, enables those workers to spend more time on the high-value task of extracting insight from the data and sharing that back with the business. This is critically important because analytics can be the driver of differentiation for any department and in the new age of self-service data preparation, the ability to act on data quickly will increasingly be the key determinant of success for many companies.