-Adam Sharman, Senior Partner, Dsifer

Organisations are collecting more data and this trend is only set to continue as IoT adoption accelerates.  However, too often, this valuable resource is locked away in silos, inaccessible to the people who need it most. These data silos can stifle innovation, hinder collaboration, and prevent organisations from fully realising the potential of their data.

In this article, why data siloes form and how to overcome them to create a more data-driven and agile organisation.

Data silos are isolated pockets of data within an organisation that are segregated from the rest of the data ecosystem. These silos can emerge for various reasons, including departmental boundaries, incompatible data formats, or data stored in separate systems that don’t communicate effectively. When data is trapped within these silos, it can’t be easily shared, analysed, or leveraged for making informed decisions.

Several factors contribute to the formation of data silos within organisations:

Lack of Data Governance: Without a robust data governance framework in place, there can be confusion regarding data ownership, definitions, and quality standards, allowing multiple data repositories to form with multiple governance approaches which can exacerbate data silos.

Departmental Barriers: Different departments within an organisation often maintain their own data repositories and systems. This division can lead to data silos, as departments may be hesitant to share their data due to concerns about data security, ownership, or privacy.

Legacy and Rogue Systems: Outdated technology and legacy systems may not support modern data integration and sharing. This can result in data becoming stuck in older systems, inaccessible to newer, more agile solutions.

Rogue systems are those that sit outside the organisation’s formal governance framework and, as such, are typically not integrated in to the organisation’s data architecture.

Cultural Barriers: In some cases, a culture of information hoarding can develop, where departments or individuals may be unwilling to share data for fear of losing control or relevance.

Data silos can significantly impede an organisation’s performance and productivity through poor decision quality resulting from incomplete information, negative customer experience resulting from disconnected service, bloated cost of IS and data storage resulting from duplicated systems and culture impacts resulting from frustration and lack of trust between teams.

All of which has a real impact to an organisation’s bottom. In fact, Gartner reported that every year in the US, poor data quality costs organisations an average $12.9 million.

To address data siloes, a multi-pronged approach must be employed that focuses on technology, governance and culture.

1. Technology

An organisation’s technology ecosystem must be designed to avoid the creation of data silos. Key to this approach is the centralisation of data in to a data warehouse supported by a robust extract, transform, load protocol. This approach ensures data availability across business departments and systems, reduces storage and processor costs, simplifies data validation and maintenance and allows for legacy or rogue system decommissioning without impacting data availability.

2. Governance

Robust and clearly defined data governance frameworks are essential to removing data siloes as they define naming, ownership, format access and data quality standards for the organisation’s data.

Effective data governance is well documented, clearly communicated and systematised (e.g. through role profiles) to ensure adoption and compliance to the standard.

3. Culture

Whilst in an organisation’s data may be where the disconnect is observed, data siloes are often just the symptom of a siloed organisational culture in which lack of trust between teams or organisation levels is pervasive.

In these cultures, protectionism can be a strong motivator to resist collaboration, including with respect to data. In order to address data siloes, the organisation must, at the same time, address any trust or lack of collaboration issues in its culture. Whilst this dimension of addressing data siloes is potentially the hardest to transform, a good starting point is typically to build empathy between teams by collaboratively mapping out a data flow end to end.

This activity highlights where each team is reliant on data from the other to be successful. We have facilitated a lot of these types of discussions and there are always some lightbulb moments.

Removing and or avoiding the creation of data siloes is typically an activity that an organisation knows should be prioritised but is often put in the ‘too hard basket’, especially as the focus goes to exciting new systems.

However, the impact of data siloes should not be underestimated and, building a robust data architecture across the three dimensions described above can avoid significant cost, time and effort in both the short term and be a competitive advantage as it accelerates a technology roadmap in the long term.