Data Discovery’s Next Chapter

The next chapter in data discovery will see an explosion in analytics globally – from tens of thousands of data analysts today to tens of thousands of business users within five years. The key drivers will be further advances in user-friendly interfaces, marriage of data discovery and unified information access (UIA), and major mobile capabilities. This is in line with the shift in enterprise spending from legacy database architectures with rigorous historical reporting to real-time event-driven, in-memory analytics platforms that accelerate and improve decision making.

As a result of their flexibility, these tools now also take the lead in providing direct integration with big data platforms such as Hadoop and Cassandra. They are characterized by beautiful presentation interfaces built on top of a new infrastructure foundation that helps non-technical users see and interpret data more easily. Established suppliers address this gap in the stack of new products coming to the market. While big sellers continue to dominate overall market share on the BI platform, the momentum experienced by data discovery vendors marks the beginning of a long-term shift in market share that will also destroy the price umbrella enjoyed by the old BI solutions.

A key value proposition for data discovery tools is that users can deploy the technology without reliance on IT over days or weeks, as opposed to up to 18 months for traditional BI tools. This faster path to user productivity – with real-time current data as opposed to stale, inflexible data sets – is a key differentiator in lowering TCO.

The dawn of self-service analysis

Unlike traditional BI solutions built on hierarchical query of pre-aggregated datasets defined by IT, data discovery tools put the entire dataset into RAM, allowing users to interact with data and ask queries based on the way their minds work. The result is a more insightful analysis that is unique to each user’s specific needs.

As the cost of RAM decreases with the transparency of 64-bit computing, memory analysis will be feasible for many organizations. With newer OLAP architectures in memory, users can run much more sophisticated real-time data analysis applications, as they could with traditional multidimensional OLAP architectures using conventional relational databases.

Facilitating access to a wide range of data types without restrictive metadata layers that allow users to conduct real-time searches and drive deeper, more valuable insights will make the return on investment compelling. Users can search associatively and define and create visualizations of the data in formats they prefer. This custom approach to analysis breaks down one of the barriers to adoption of traditional BI solutions and opens the market to a much larger potential user community.

As these users search for on-the-fly data feeds and statistical analysis, they can expect newer features on BI platforms to focus on predictable analytics models and forecasting algorithms that are easier to consume in dashboards. Also, expect to see the integration of UIA that combines the best of both search and BI capabilities. UIA software allows users to create a mini instant data warehouse by unifying access to multiple types and sources of both structured and unstructured information in a single archive called a database table. Users can then perform searches and generate BI-type reports.

UIA technology replaces large legacy enterprise search apps because of its ability to provide a single overview of all information. Another related shift will be the reuse of traditional data warehouses, which will eventually only be used for data that is not frequently requested.

Finally, mobile BI has the potential to significantly expand the user population. Over the next year, we are seeing significant progress in facilitating the delivery of BI applications to a more user-friendly and mobile device.

For suppliers, a triple go-to-market strategy

Business users are becoming more and more influential in purchasing data discovery solutions with user-friendly transitioning functionality as the primary BI procurement criteria. Many leave notoriously complex and inflexible traditional BI platforms and bypass IT by purchasing data discovery tools that offer a faster, easier and more efficient way to model, navigate and visualize data in their large data warehouses.

While this may jeopardize the creation of fragmented data silos, it has also dramatically increased the average number of users per user. Implementation. In the end, IT becomes involved by providing architectures, methodologies and information management policies that bridge the gap between legacy BI and data discovery solutions.

The go-to-market strategy for data discovery vendors must consist of a three-pronged approach that addresses specific use cases or pain points (read: sales sales). Sellers must be flexible in how the customer wants to consume the product, whether in the cloud or on-site. With this in mind, the following strategies range from large enterprise-driven solutions to departments and SMEs.

  1. Direct sales aimed at large companies with coordinated targeting of CIOs and departments / business units that gain more control over analytical budgets, making them more influential buyers;
  2. Partnerships with channel partners, such as BI practice areas with system integrators, resellers who have made a strategic commitment to BI tools, and the advanced technology groups of major distributors who will invest behind the product;
  3. A cloud-based “freemium” model that can be effectively serviced in sales. As the number of subscriptions to an account grows, the model can be scaled across the company.

This balanced, flexible approach will accelerate adoption by expanding market coverage. It will also help to buffer the impact of operating margin on ramp distribution and customer acquisition.