The noble effort to build a “data-centric” culture is really a journey, not a destination. With that perspective, we can understand that no matter how good a given environment seems to be –especially compared to whatever existed before – there’s always room for enhancement. As more technologies, strategies, and disciplines emerge, the ongoing evolution ensures constant improvement. And by the way, we’re not even close to real data-centricity yet, but at least we’re getting closer. The growing discipline around data mesh architectures represents a perfect milestone in this endless odyssey. There’s progress here, for sure, but it’s just as important to keep looking ahead. 

Data mesh offers a (relatively) fresh approach in that the focus is on the data itself, rather than the data lake or data warehouse resources and pipelines that move and/or store it. This strategy is founded on a federated data model – the data architecture is organized to meet the needs of diverse business domains, and ownership is assigned to domain-specific teams instead of a central authority. With a data mesh model, data is a product that’s more easily accessible to appropriate constituencies, not hoarded by a few select parties. Among other benefits, this eases scaling and analytics within larger organizations that have heterogeneous infrastructures. 

There’s a lot to like about data mesh, and with a wide-angle lens we can see that it illuminates the power of data itself. It’s no longer an abstract concept or a by-product of different apps and other technologies; used appropriately and purposefully, data delivers true value while fueling innovation and optimal decision-making.

The process behind these benefits is also noteworthy. Most importantly, the notion of data decentralization is deceptively simple, and potentially revolutionary. Think of how IT consumerization has upended traditional technology implementation: Where IT specialists once made all the decisions on which tools to buy for business professionals and dictated how all that hardware and software was to be used, those end users now call the shots. They freely buy the devices they want and download the apps they like, then wait for IT to catch up. This provides enormous benefits (perhaps with headaches related to security, integration, and support). 

With data mesh we’re seeing similar movement toward data democratization. When line-of-business teams and other constituencies within the enterprise gain unprecedented access, and even ownership, of business data that was previously guarded, it accelerates collaboration and enables custom strategies to solve specific business problems. Data access also becomes simpler when interfaces and navigation are not just user-friendly but attuned to the priorities of specific functions, rather than having a more generic or enterprise-wide approach. 

And hopefully, everyone also understands (or should understand) that autonomy brings with it a healthy level of responsibility. A Wild West mentality is not advisable – each domain-centric or other appropriate team must develop standards for data quality, establish and enforce usage policies, and ensure compliance. (If not, they could lose control.)

Data Mesh 2.0: True Collaboration

Of course, no single technology approach is a panacea, and even the best solutions have downsides. Data mesh is no exception. 

For one thing, the praise for decentralization must be balanced with concerns over fragmentation. When corporate data – especially sensitive material subject to privacy mandates and other regulations – is scattered across diverse systems, databases, and formats, the level of complexity is instantly higher. Also, it’s not as if one particular dataset always belongs only to one particular constituency; different departments often need the same data for different purposes, in different formats and under different regulatory constraints. Even when the process is handled appropriately, it can be a governance nightmare – think problems related to duplication, versioning, integration, security, scalability, maintenance, and a lot more. 

In fact, all of this fragmentation can take us back to an old problem: data silos. When individual departments or multi-disciplinary teams hoard their data, the same way corporate data stewards used to with all internal data, it undermines the very spirit of collaboration and data ownership. 

Again, the benefits of the decentralized architecture enabled by data mesh are undeniable, and data mesh does get us closer to a data-centric environment. But we’re still a long way away from that nirvana.

In fact – and it sounds harsh to say this – data mesh is still so much more potential than reality that it almost counts as vaporware. Comprehensive use cases are hard to come by, and even successful deployments have a relatively narrow focus. 

So, what will it take for data mesh to deliver on its promise? Some points are pretty basic but still vital, and not just for data mesh. For example, senior management (the C-suite, LOB heads, etc.) have to get involved, and make that involvement more visible – the rank-and-file employee base should see how executives are making data-driven decisions. It would also help to ramp up employee training around data literacy, because it’s hard to get a real mindset change without accompanying elevation in skillsets. Finally, even with a decentralized architecture, the organization as a whole will benefit from transparent governance and standard-setting around data quality and security. 

But even this only moves us a little further. We still need data models that essentially restructure the entire ecosystem. We’ve had far-reaching advances in data generation, data dissemination, data storage, and more. Yet when it comes to data integration, we’re still tethered to operating practices that haven’t evolved in decades – and which can soak up half the IT budget. 

What we need is data that’s integrated without data integration. That will help us reach the real promise of a data-centric environment, which is secure, compliant, and speedy data-enabled collaboration. In an operating universe with massive data volumes, a steady stream of new mandates and many other pressures, this is already becoming a necessity, not some distant fantasy.

Maybe it’s time for mesh-plus. Data can no longer be seen as a key element, as it is with data mesh; it must actually be decoupled from the applications and other technologies used to create and store it. By freeing the gift from the wrapping, we can establish data as its own network. Despite all its other benefits, most data mesh implementations still aren’t being used to operationally manage data and create data products that are federated. This isn’t just around the corner, it’s already achievable with new technologies.   

These advances also allow the development of what we can call a self-serve data platform – a gateway to real collaboration with the power to discover, access, change, and even originate data in ways we can’t do now. Again, this is on top of existing data mesh architecture, and without creating new silos. We also get federated computational governance; by transferring Data Governance policies from domain owners to the data itself, we ensure that permissions, controls, rules, and more stay consistent no matter how the data is accessed.

Emerging technologies that move the ball forward should be welcomed, especially as they get us closer to the so-far elusive goal of a data-centric culture. Data mesh is clearly in that category. However, meaningful progress requires a major transformation in long-held business practices and a readiness to embrace constant change. The effort really is worth it.