Shaping the strategy

Our engagement began with strategic workshops aimed at shaping a new data management strategy centered around the principles of data mesh. These sessions were designed to establish common understanding, shape the strategy, and get executive committee approval.

Engaging stakeholders

Throughout the journey, we actively engaged with various stakeholders across the organization to ensure alignment and buy-in. We did management consulting for the overall strategy with data leaders such as the Chief Data Officer (CDO), Heads of Data and Governance departments, IT Leads, etc, all the way towards implementation and technology with Architects, Lead Data Engineers, Data Governance Leads, etc. Effectively riding the architect elevator to translate strategic directions from the penthouse into concrete solutions in the engine room and vice versa.

Defining the Data Product

To ensure clarity and common understanding, we focused on defining what a data product is and what it is not. This helped in evangelizing the data mesh strategy across the bank.

Architectural vision and self-service data platform

It quickly became clear that the primary challenge was the technical conceptualization of Data Products. Therefore, our initial focus went to clarifying and concretizing Data Products from a technical perspective. 

To reduce the cognitive load required for data product developers to create and share valuable data, we developed a comprehensive architectural vision and blueprint for a self-service data platform. This vision centered on the concept of a data product, an independently deployable and manageable unit containing data, data transformation code, and metadata. By abstracting away the underlying complexities of data infrastructure and technologies, the data product model empowers domain experts to own and serve their data, fostering autonomy and accelerating data-driven innovation across the bank. This approach shifts the paradigm from centralized data management to a decentralized data mesh, promoting agility, scalability, and improved data quality through decentralized ownership and accountability. 

Our approach was to complement the existing technology stack as much as possible to leverage previous investments and consider the data product and its self-service data platform as the core glue elevating a set of individual technologies into an integrated self-service experience.

The bank's data mesh implementation began with analyzing their current tech landscape and cataloging existing data technologies. A future technology stack was designed to support independent data products, focusing on self-service and domain ownership through a shared platform, including technologies for storage, processing, querying, security, and governance amongst others. We then defined the Minimum Viable Platform's technology components to enable a sound foundation and ease of use, ensuring alignment between strategy and implementation. 

This effort resulted in officially validated reference documents and blueprints on:

• Data product definition
• Architectural vision for the self-service data platform with supporting technology stack
• Positioning of existing and new architectural patterns with respect to this new solution

Proof of concept

We implemented a technical proof of concept of the self-service data platform to assess its feasibility within the complex IT landscape of the bank. The proof of concept was strategically designed to address and mitigate the highest risks early on, ensuring a clear and realistic understanding of what it would take to successfully realize the platform. As part of this initiative, we developed a functional platform orchestrator leveraging Kubernetes as a universal control plane. This orchestrator seamlessly integrated with the existing technology stack and automated key processes across the data product lifecycle. For example:

• It provisioned for each data product output port the necessary schemas directly in the data storage solution.
• It created for each data product output port the required buckets in the object storage solution.
• It registered essential data product metadata in the data catalog and marketplace for discovery.
• It built, deployed and executed the transformation code within a scalable compute solution.
• It enabled uniform access and querying through a query engine solution.
• …

As such, our platform engineering approach for data mesh is bringing the core principles of Data Mesh to life in a real-world setting. The hands-on experience of ACA Group in building data mesh platforms at multiple customers bootstrapped their journey.

Guiding the Data Mesh Program

With the success of strategic consulting, an architectural vision based on our data mesh experience, and a concrete proof of concept, ACA Group helped guide and steer the data mesh program across multiple working tracks. Because data mesh is often perceived as too abstract, each track was established to make it as concrete as possible along 3 axes: 

• business value – impact on business, 
• technical architecture – impact on technology, 
• people and processes – impact on the way of working, governance, roles, responsibilities and policies. 

Data Product Landscape Track

Focused on discovering and identifying potential data products to make the business value and roadmap concrete for business and other stakeholders.

Self-Service Data Platform Track

Focused on architectural vision, technology stack setup and proof of concept to make the technical implications and solution concrete for technical stakeholders.

Federated Governance and Operating Model Track

Focused on clarifying team roles and responsibilities to make the people and culture impact concrete for stakeholders. Next to the more general governance and operating model for data mesh, this also included more detailed work on data product modeling guidelines linked to the enterprise and domain data models, governance challenges to solve and for which policies need to be embedded into the platform, and shaping the metadata strategy around data products through suitable data product specifications and data contracts.

ACA Group is instrumental from strategy to implementation

This comprehensive approach is instrumental for the bank's data mesh journey. We injected a clear foundational vision that goes beyond the theoretical or abstract concepts of Data Mesh. For this, we worked closely with data leaders within the bank to shape the data management strategy for years to come, and we made it concrete by collaborating with architects, data engineers and technology experts. ACA Group helped this Belgian bank in effectively shaping and translating a data mesh strategy into a tangible minimal implementation allowing for further growth through a self-service platform, decentrally owned data products, federated governance and operating model, and a mesh of data products delivering business value.