Big data and analytics have climbed to the top of the corporate agenda as organizations seek to become nimbler by applying data-driven decision-making techniques to the senior management toolkit. The promise is that the insights gained from big data and analytics will transform the way companies do business, delivering the kind of performance gains last seen in the 1990s, when organizations redesigned their core processes. Data-driven strategies provide a way for organizations that are not net-native, as well as those that are, to better compete in the digital economy by being able to develop more responsive business processes.

This positive momentum around big data has encouraged organizations to expand their business intelligence (BI) initiatives beyond traditional enterprise applications by embracing previously ignored data formats (including unstructured data such as video, audio and images) and new data sources (including mobile applications, social media and customer engagement channels). In addition, businesses have woken up to what is possible via the myriad sensors, embedded computers, industrial controllers and connected devices (such as vehicles, wearable computing devices, robots and drones) that make up the emerging Internet of Things.

As a result, IT departments are being asked not just to store data in warehouses and data lakes, but to provide the infrastructure to perform analytics on it as well as to create the algorithms and models that agile business requires. However, enterprises that are delivering data to their business lines based on their investment in databases, data warehouses, analytics appliances and ETL tools are struggling to harness big data and advanced analytics because of the volume and types of data involved. More people in the organization want to use data immediately for one-off projects, regulatory auditing and ‘what if’ scenarios for data scientists. Many CIOs have invested in creating data lakes using Hadoop distributions, but continue to struggle to deliver the actionable data businesses want.

Developing an enterprise data marketplace or bazaar that uses a modern data management architecture for data that exists on a hyperscale platform, and enables a wide variety of end-user access via self-service, on-demand models, would seem to be the answer. Delivering this capability is difficult, however, and that is where IT service providers can help.

This Technology & Business Insight report on the development of the Enterprise Data Bazaar is based 40 primary research interviews undertaken by 451 Research with IT service providers, technology vendors and enterprise buyers in the second half of 2017. The full report includes:

• The best approach for modern data management is to concentrate on building a governance layer on top of an existing data lake. 
• The starting point of an enterprise data bazaar or marketplace.
• The evaluation and importance for organizations to have a chief data officer to champion a company-wide strategy for the capture, management and sharing of data.
• And the differences between governance and security policies across data source systems create challenges for companies when it comes to auditing data in lakes. Particularly as requirements around personally identifiable information and the General Data Protection Regulation go into effect, the need for more easily audited data can be a catalyst for the data bazaar.